-- Logs begin at Mon 2022-08-08 01:09:49 UTC, end at Mon 2022-08-08 02:46:29 UTC. -- Aug 08 02:16:51.276092 dv-cinder-740146-18-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Aug 08 02:16:51.283371 dv-cinder-740146-18-iscsi devstack@n-api.service[103870]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Aug 08 02:16:51.289212 dv-cinder-740146-18-iscsi devstack@n-api.service[103870]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Aug 8 02:16:51 2022] *** Aug 08 02:16:51.289212 dv-cinder-740146-18-iscsi devstack@n-api.service[103870]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 08 02:16:51.289212 dv-cinder-740146-18-iscsi devstack@n-api.service[103870]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 08 02:16:51.289212 dv-cinder-740146-18-iscsi devstack@n-api.service[103870]: nodename: dv-cinder-740146-18-iscsi Aug 08 02:16:51.289212 dv-cinder-740146-18-iscsi devstack@n-api.service[103870]: machine: x86_64 Aug 08 02:16:51.289212 dv-cinder-740146-18-iscsi devstack@n-api.service[103870]: clock source: unix Aug 08 02:16:51.289212 dv-cinder-740146-18-iscsi devstack@n-api.service[103870]: pcre jit disabled Aug 08 02:16:51.289212 dv-cinder-740146-18-iscsi devstack@n-api.service[103870]: detected number of CPU cores: 4 Aug 08 02:16:51.289212 dv-cinder-740146-18-iscsi devstack@n-api.service[103870]: current working directory: / Aug 08 02:16:51.289212 dv-cinder-740146-18-iscsi devstack@n-api.service[103870]: detected binary path: /usr/bin/uwsgi-core Aug 08 02:16:51.289212 dv-cinder-740146-18-iscsi devstack@n-api.service[103870]: your processes number limit is 47680 Aug 08 02:16:51.289212 dv-cinder-740146-18-iscsi devstack@n-api.service[103870]: your memory page size is 4096 bytes Aug 08 02:16:51.289212 dv-cinder-740146-18-iscsi devstack@n-api.service[103870]: detected max file descriptor number: 2048 Aug 08 02:16:51.289212 dv-cinder-740146-18-iscsi devstack@n-api.service[103870]: lock engine: pthread robust mutexes Aug 08 02:16:51.289212 dv-cinder-740146-18-iscsi devstack@n-api.service[103870]: thunder lock: enabled Aug 08 02:16:51.289212 dv-cinder-740146-18-iscsi devstack@n-api.service[103870]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Aug 08 02:16:51.289212 dv-cinder-740146-18-iscsi devstack@n-api.service[103870]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 08 02:16:51.327643 dv-cinder-740146-18-iscsi devstack@n-api.service[103870]: Python main interpreter initialized at 0x559cff3899c0 Aug 08 02:16:51.327643 dv-cinder-740146-18-iscsi devstack@n-api.service[103870]: python threads support enabled Aug 08 02:16:51.327643 dv-cinder-740146-18-iscsi devstack@n-api.service[103870]: your server socket listen backlog is limited to 100 connections Aug 08 02:16:51.327643 dv-cinder-740146-18-iscsi devstack@n-api.service[103870]: your mercy for graceful operations on workers is 90 seconds Aug 08 02:16:51.327972 dv-cinder-740146-18-iscsi devstack@n-api.service[103870]: mapped 403077 bytes (393 KB) for 2 cores Aug 08 02:16:51.328064 dv-cinder-740146-18-iscsi devstack@n-api.service[103870]: *** Operational MODE: preforking *** Aug 08 02:16:51.328289 dv-cinder-740146-18-iscsi devstack@n-api.service[103870]: *** uWSGI is running in multiple interpreter mode *** Aug 08 02:16:51.328289 dv-cinder-740146-18-iscsi devstack@n-api.service[103870]: spawned uWSGI master process (pid: 103870) Aug 08 02:16:51.328484 dv-cinder-740146-18-iscsi systemd[1]: Started Devstack devstack@n-api.service. Aug 08 02:16:51.329058 dv-cinder-740146-18-iscsi devstack@n-api.service[103870]: spawned uWSGI worker 1 (pid: 103871, cores: 1) Aug 08 02:16:51.329386 dv-cinder-740146-18-iscsi devstack@n-api.service[103870]: spawned uWSGI worker 2 (pid: 103872, cores: 1) Aug 08 02:16:51.329531 dv-cinder-740146-18-iscsi devstack@n-api.service[103870]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 08 02:16:51.992488 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Aug 08 02:16:51.993092 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Aug 08 02:16:53.100866 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 08 02:16:53.100866 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: from cryptography.utils import int_from_bytes Aug 08 02:16:53.101475 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 08 02:16:53.101475 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: from cryptography.utils import int_from_bytes Aug 08 02:16:53.122131 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 08 02:16:53.122131 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: from cryptography.utils import int_from_bytes Aug 08 02:16:53.122688 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 08 02:16:53.122688 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: from cryptography.utils import int_from_bytes Aug 08 02:16:55.637608 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 08 02:16:55.638233 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 08 02:16:55.638374 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 08 02:16:55.638735 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 08 02:16:55.638992 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 08 02:16:55.639379 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.639706 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.640027 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.640345 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.640693 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.641009 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.641330 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.641652 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.641974 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.642073 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 08 02:16:55.642432 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.642521 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 08 02:16:55.642605 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.642760 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 08 02:16:55.642867 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-740146-18-iscsi {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.643118 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 08 02:16:55.643216 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.643370 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 08 02:16:55.643462 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.643746 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.643885 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.644062 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.644192 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.644363 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.644493 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.644693 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.645329 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.645520 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.646023 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.646174 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.646336 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.646455 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.646660 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.646777 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.646980 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.647097 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.647328 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.647497 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.647634 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.647753 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.647881 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.648008 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.648179 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-740146-18-iscsi {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.648305 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.648498 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.648633 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.648827 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.648960 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.649141 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.649276 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.649457 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.649587 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-740146-18-iscsi {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.649760 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.649895 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.650078 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.650202 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.650383 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.650512 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.650883 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.651017 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.651187 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.651324 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.651490 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.651646 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.651797 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.651924 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.652114 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.652243 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.652424 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.652557 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.652762 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.652897 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.653078 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.653213 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.653383 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.653517 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.653700 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.653830 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.654001 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.654139 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.654307 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.654439 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-740146-18-iscsi {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.654614 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.654746 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.654978 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.655284 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.655688 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.655996 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.656300 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.656616 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.656778 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.656968 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.657100 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.657275 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.657413 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.657597 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.657730 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.657912 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.658043 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.658228 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.658801 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.659106 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.659242 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.659411 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.659540 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.659716 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.659842 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.660016 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.660140 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.660260 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.660381 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.661212 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.661513 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.661675 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.661830 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.661962 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.662240 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.662564 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.662886 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.663210 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.663463 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.663713 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.664032 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.664352 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.664567 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.664773 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.664949 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.665268 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.665394 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.665591 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.665847 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.665979 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.666166 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.666303 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.666479 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.666614 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.666789 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.666914 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.667096 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.667235 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.667404 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.667529 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.667724 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.667860 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.668038 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.668178 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.668357 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.668719 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.669000 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.669392 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.27 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.669643 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.27 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.669955 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.670294 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.670610 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.670915 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.671153 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.27 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.671306 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.671440 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.27 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.671733 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.671863 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.672086 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.672211 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.672414 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.672540 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.672734 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.672860 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.673051 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.673175 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.673377 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.673498 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.673669 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.673792 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.673913 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.674042 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.674200 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.674326 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.674499 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.674634 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.674811 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.674952 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.675111 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.675236 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.675411 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.675535 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.675709 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.675846 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.676004 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.676130 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.676463 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.676804 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.676932 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.677117 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.677252 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.677426 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.677551 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.677730 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.677862 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.678038 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.678163 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.678346 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.678470 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.678653 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.678787 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.678961 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.679087 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.679268 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.679504 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.679629 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.679828 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.679953 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.680151 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.680282 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.680454 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.680583 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.680780 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.680904 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.681084 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.681221 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.681389 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.681520 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.681958 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.682097 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.682266 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.682395 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.682570 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.682697 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.682884 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.683007 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.683186 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.683314 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.683486 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.683610 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.683799 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.683924 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.684100 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.684224 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.684401 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.684526 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.684736 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.684864 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.685036 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.685160 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.685335 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.685459 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.685650 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.685785 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.685952 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.686078 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.686260 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.686393 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.686576 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.686709 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.686882 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.687013 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.687186 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.687319 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.687489 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.687622 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.687789 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.687918 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.688091 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.688223 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.688396 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.688718 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.689026 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.689327 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.689451 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.689644 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.689778 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.689963 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.690087 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.690271 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.690413 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.690578 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.690884 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.691202 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.691471 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.691595 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.691880 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.692221 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.692344 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.692538 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.692719 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:16:55.692896 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.693038 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.693227 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.693352 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.693551 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.693676 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.693864 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.694174 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.694310 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.694497 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.694622 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.694809 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.694947 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.695120 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.695252 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.695430 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.695562 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.695746 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.696064 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.696254 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.696385 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.696587 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.696740 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.696926 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.697066 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.697241 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.697380 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.697557 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.697683 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.697875 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.698009 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.698259 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.698579 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.698890 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.699027 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.699230 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.699370 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.699574 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.699700 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.699885 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.700011 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.700209 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.700335 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.700527 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.700709 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.700863 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.700989 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.701175 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.701311 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.701486 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.701611 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.701807 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.701918 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.702325 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.702462 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.702576 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.702705 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.702828 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.702956 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.703085 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.703218 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.703340 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.703461 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.703640 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.703775 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.703952 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.704076 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.704261 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.704386 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.704591 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.704744 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.704949 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.705073 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.705260 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.705391 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.705576 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.705710 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.705886 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.706012 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.706196 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.706322 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.706504 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.706637 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.706810 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.706941 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.707111 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.707234 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.707939 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.708248 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.708424 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.708559 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.708715 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.708852 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.709009 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.709176 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.709325 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.709501 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.709633 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.709806 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.709939 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.710138 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.710270 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.710431 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.710557 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.710741 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.710869 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.711050 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.711175 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.711358 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.711482 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.711661 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.711797 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.711969 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.712102 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.712280 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.712404 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.712638 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.712798 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.712939 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.713061 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.713227 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.713365 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.713530 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.713655 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.713837 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.713968 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.714148 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.714272 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.714452 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.714578 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.714753 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.714877 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.715060 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.715184 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.715367 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.715498 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.715667 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.715799 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.715976 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.716110 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.716283 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.716415 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.716587 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.716758 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.716919 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.717058 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.717220 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.717345 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.717520 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.717653 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.717833 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.717967 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.718141 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.718266 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.718450 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.718574 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.718758 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.718891 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.719060 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.719189 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.719369 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.719494 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.719673 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.719799 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.719976 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.720101 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.720279 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.720418 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.720581 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.720721 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.720900 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.721037 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.721204 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.721340 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.721508 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.721634 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.721812 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.721938 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.722116 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.722241 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.722423 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.722555 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.722746 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.722881 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.723603 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.723796 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.723933 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.724197 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.724371 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.724542 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.724713 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.724877 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.725013 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.725218 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.725350 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.725554 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.725679 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.725874 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.726000 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.726173 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.726297 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.726494 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.726619 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.726819 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.726942 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.727089 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.727222 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.727390 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.727525 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.727690 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.727816 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.727999 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.728129 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.728316 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.728448 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.728624 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.728770 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.728968 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.729093 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.729280 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.729416 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.729602 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.729728 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.729907 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.730030 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.730219 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.730352 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.730522 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.730653 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.730827 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.730950 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.731135 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.731262 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.731443 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.731569 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.731760 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.731884 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.732067 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.732195 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.732379 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.732503 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.732702 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.732855 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.733055 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.733186 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.733309 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.733430 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.733615 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.733741 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.733919 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.734044 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.734230 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.734355 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.734536 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.734658 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.734852 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.734979 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.735159 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.735284 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.735480 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.735615 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.735786 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.735911 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.736097 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.736229 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.736397 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.736529 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.736719 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.736856 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.737026 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.737151 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.737331 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.737471 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.737645 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.737770 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.737949 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.738074 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.738262 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.738399 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.738601 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.738730 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.738920 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.739358 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.739497 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.739670 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.739796 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.739980 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.740106 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.740280 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.740412 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.740592 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.740759 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.740927 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.741050 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.741239 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.741375 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.741544 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.741669 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.741849 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.741972 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.742154 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.742290 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.742458 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.742583 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.742761 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.742879 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.743105 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.743247 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.743370 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.743491 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.743649 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.743775 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.743960 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.744096 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.744273 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.744400 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.744579 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.744728 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.744906 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.745031 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.745206 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.745333 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.745516 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.745639 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.745817 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.745941 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.746116 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.746270 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.746415 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.746538 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.746720 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.746844 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.747034 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.747158 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.747337 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.747472 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.747652 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.747774 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.747963 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.748086 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.748262 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.748395 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.748559 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.748729 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.748895 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.749017 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.749195 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.749331 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.749494 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.749605 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.749792 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.749927 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.750091 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.750226 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.750412 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.750542 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.750720 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.750845 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.751020 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.751144 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.751326 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.751448 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.751641 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.751764 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.751948 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.752072 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.752254 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.752376 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.752559 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.752706 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.752888 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.753004 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.753198 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.753333 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.753525 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.753681 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.753814 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.754090 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.754250 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.754604 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.754922 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.755065 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.755242 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.755367 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.755555 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.755686 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.755874 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.756070 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.756200 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.756320 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.756477 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.756601 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.756813 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.756950 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.757117 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.757252 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.757427 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.757552 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.757743 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.757876 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.758056 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.758302 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.758429 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.758631 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.758755 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.758943 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.759079 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.759247 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.759400 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.759559 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.759685 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.759864 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.759990 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.760168 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.760301 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.760471 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.760627 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.760784 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.760907 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.761123 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.761259 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.761409 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.761543 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.761711 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.761844 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.762012 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.762146 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.762317 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.762440 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.762618 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.762756 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.762921 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.763046 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.763222 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.763357 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.763522 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.763655 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.763819 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.763943 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.764119 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.764246 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.764419 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.764553 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.764723 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.764847 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.765043 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.765181 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.765347 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.765473 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.765653 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.765779 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.765951 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.766084 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.766270 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.766396 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.766573 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.766699 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.766871 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.766998 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.767168 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.767301 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.767465 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.767591 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.767766 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.767889 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.768075 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.768208 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.768382 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.768508 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.768707 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.768854 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.769018 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.769143 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.769320 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.769452 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.769627 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.770287 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.770478 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.770783 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.770938 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.771089 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.771262 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.771396 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.771578 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.771710 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.771888 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.772020 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.772203 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.772329 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.772514 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.772640 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.772867 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.772989 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.773144 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.773279 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.773455 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.773588 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.773755 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.773886 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.774049 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.774171 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.774349 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.774482 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.774655 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.774787 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.774957 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.775117 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.775250 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.775370 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.775559 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.775684 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.775856 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.775987 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.776138 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.776260 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.776445 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.776577 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.776769 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.776898 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.777070 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.777240 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.777364 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.777492 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.777662 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.777787 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.777971 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.778094 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.778272 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.778411 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.778570 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.778703 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.778920 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.779051 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.779181 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.779310 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.779458 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.779591 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.779756 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.779879 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.780082 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.780205 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.780348 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.780472 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.780648 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.780808 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.780969 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.781111 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.781269 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.781387 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.781565 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.781688 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.781867 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.781993 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.782173 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.782305 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.782470 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.782601 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.782773 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.782907 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.783072 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.783212 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.783373 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.783505 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.783677 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.783809 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.783979 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.784104 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.784277 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.784410 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.784580 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.784748 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.784900 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.785038 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.785198 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.785334 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.786046 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.786177 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.786419 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.786723 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.786846 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.787039 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.787165 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.787350 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.787476 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.787662 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.787795 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.787966 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.788090 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.788270 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.788395 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.788587 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.788753 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.788924 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.789050 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.789236 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.789362 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.789556 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.789691 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.789877 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.790009 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.790199 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.790331 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.790501 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.790635 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.790806 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.790938 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.791114 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.791244 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.791421 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.791546 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.791720 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.791851 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.792026 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.792157 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.792331 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.792462 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.792637 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.792781 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.792960 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.793094 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.793265 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.793385 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.793572 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.793696 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.793880 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.794005 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.794184 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.794316 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.794487 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.794611 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.794788 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.794926 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.795108 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.795241 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.795406 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.795531 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.795707 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.795839 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.796009 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.796131 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.796308 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.796431 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.796610 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.796785 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.796917 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.797038 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.797232 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.797356 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.797540 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.797678 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.797849 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.797974 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.798154 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.798279 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.798461 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.798587 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.798763 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.798896 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.799068 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.799193 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.799372 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.799498 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.799673 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.799799 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.799993 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.800120 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.800298 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.800431 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.800598 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.800778 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.800926 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.801071 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.801326 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.801695 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.801820 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.802013 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.802140 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.802318 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.802444 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.802632 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.802754 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.802935 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.803061 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.803237 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.803363 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.803557 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.803685 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.803866 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.803999 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.804184 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.804310 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.804512 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.804635 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.804849 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.804974 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.805150 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.805284 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.805558 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.805691 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.805865 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.805999 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.806171 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.806304 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.806477 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.806603 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.807039 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.807175 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.807338 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.807470 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.807642 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.807768 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.807956 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.808088 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.808253 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.808377 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.808551 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.808690 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.808893 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.809018 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.809199 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.809337 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.809496 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.809619 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.809814 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.809936 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.810112 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.810238 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.810433 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.810567 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.810749 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.810882 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.811072 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.811207 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.811460 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.811584 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.811764 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.811893 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.812080 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.812206 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.812395 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.812520 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.812730 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.812861 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.813033 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.813160 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.813334 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.813458 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.813645 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.813764 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.813957 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.814083 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.814262 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.814389 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.814570 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.814702 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.814874 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.815005 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.815173 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.815296 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.815478 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.815611 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.815786 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.815911 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.816087 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.816213 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.816389 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.816522 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.817292 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.817675 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.817815 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.818197 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.818329 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.818520 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.818655 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.818850 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.818996 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.819168 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.819303 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.819477 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.819603 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.819799 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.819924 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.820109 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.820242 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.820411 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.820567 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.820721 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.820858 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.821032 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.821236 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.821368 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.821490 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.821610 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.821731 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.821919 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.822058 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.822224 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.822350 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.822524 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.822651 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.822823 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.822959 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.823139 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.823272 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.823436 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.823563 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.823753 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.823872 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.824063 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.824189 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.824386 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.824513 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.824720 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.824851 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.825023 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.825149 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.825341 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.825466 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.825643 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.825767 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.825941 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.826065 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.826242 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.826374 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.826538 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.826670 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.826838 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.826962 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.827141 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.827280 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.827442 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.827568 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.827763 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.827889 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.828067 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.828203 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.828384 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.828522 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.828714 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.828842 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.829018 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.829161 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.829323 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.829448 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.829625 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.829758 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.829946 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.830079 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.830245 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.830364 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.830544 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.830679 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.830851 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.830978 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.831151 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.831278 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.831450 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.831576 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.831750 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.831877 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.832053 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.832187 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.832960 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.833374 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.833515 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.833697 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.833816 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.834016 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.834149 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.834337 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.834465 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.834655 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.834787 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.834972 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.835106 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.835289 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.835416 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.835657 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.835812 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.835940 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.836147 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.836287 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.836466 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.836601 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.836812 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.836938 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.837130 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.837267 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.837445 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.837571 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.837761 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.837893 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.838072 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.838196 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.838382 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.838514 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.838692 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.838823 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.839003 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.839135 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.839311 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.839435 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.839623 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.839750 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.839934 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.840061 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.840242 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.840368 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.840555 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.840717 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.840891 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.841018 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.841204 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.841327 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.841518 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.841715 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.841873 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.842039 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.842172 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.842350 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.842475 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.842659 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.842786 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.842968 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.843093 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.843278 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.843416 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.843586 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.843712 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.843894 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.844019 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.844207 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.844341 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.844512 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.844639 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.844842 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.844983 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.845164 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.845297 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.845479 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.845605 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.845787 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.845911 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.846110 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.846236 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.846418 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.846541 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.846729 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.846854 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.847038 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.847176 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.847346 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.847470 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.847663 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.847788 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.848104 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.848231 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.848503 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.848795 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.848924 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.849114 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.849249 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.849430 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.849556 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.849744 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.849869 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.850055 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.850183 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.850363 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.850497 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.850672 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.850805 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.851004 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.851135 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.851259 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.851388 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.851569 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.851703 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.851872 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.851997 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.852177 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.852303 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.852487 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.852620 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.852815 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.852941 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.853137 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.853262 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.853452 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.853628 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.853754 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.853966 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.854093 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.854318 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.854557 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.854685 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.854868 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.854995 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.855196 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.855323 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.855520 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.855647 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.855842 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.855967 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.856155 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.856280 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.856459 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.856584 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.856789 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.856927 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.857091 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.857215 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.857393 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.857520 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.857694 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.857817 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.857997 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.858130 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.858298 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.858434 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.858604 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.858736 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.858908 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.859045 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.859205 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.859328 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.859504 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.859628 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.859814 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.859947 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.860124 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.860337 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.860472 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.860718 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.860862 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.860994 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.861117 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.861314 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.861447 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.861617 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.861750 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.861920 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.862047 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.862220 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.862346 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.862529 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.862664 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.862830 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.862955 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.863131 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.863257 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.863429 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.863600 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.863930 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.864263 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.864380 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.864582 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.864721 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.864917 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.865046 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.865237 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.865361 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.865541 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.865673 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.865846 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.865971 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.866160 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.866287 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.866464 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.866598 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.866792 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.866930 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.867100 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.867227 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.867405 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.867539 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.867709 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.867842 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.868018 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.868143 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.868320 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.868568 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.868724 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.868930 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.869068 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.869230 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.869364 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.869539 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.869664 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.869841 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.869967 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.870147 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.870282 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.870447 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.870572 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.870752 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.870879 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.871053 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.871193 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.871360 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.871486 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.871661 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.871786 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.871969 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.872103 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.872269 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.872393 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.872578 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.872719 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.872915 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.873043 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.873229 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.873355 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.873540 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.873666 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.873845 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.873970 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.874156 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.874280 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.874463 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.874589 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.874771 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.874898 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.875080 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.875317 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.875452 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.875656 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.875782 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.875980 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.876106 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.876315 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.876442 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.876626 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.876772 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.876966 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.877092 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.877278 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.877402 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.877587 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.877714 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.877895 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.878028 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.878206 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.878331 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.878513 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.878641 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.878820 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.878952 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.879635 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.879779 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.880068 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.880379 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.880510 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.880815 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.880943 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.881142 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.881272 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.881436 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.881563 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.881746 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.881877 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.882058 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.882183 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.882367 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.882491 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.882678 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.882816 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.882987 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.883116 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.883298 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.883423 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.883605 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.883738 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.883935 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.884069 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.884248 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.884380 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.884565 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.884723 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.884907 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.885040 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.885221 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.885353 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.885524 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.885649 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.885829 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.885961 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.886137 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.886263 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.886443 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.886577 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.886750 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.886873 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.887055 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.887181 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.887360 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.887487 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.887661 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.887784 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.887963 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.888097 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.888267 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.888403 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.888571 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.888745 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.888900 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.889040 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.889209 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.889343 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.889512 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.889638 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.889815 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.889949 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.890117 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.890254 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.890421 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.890547 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.890724 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.890859 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.891026 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.891152 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.891336 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.891462 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.891639 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.891766 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.891942 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.892067 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.892244 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.892369 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.892549 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.892705 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.892878 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.893005 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.893207 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.893333 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.893539 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.893666 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.893852 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.893985 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.894152 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.894285 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.894456 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.894581 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.894775 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.894935 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.895233 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.895727 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.895854 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.896047 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.896172 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.896360 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.896487 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.896725 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.896855 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.897009 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.897133 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.897314 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.897452 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.897620 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.897747 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.897927 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.898052 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.898233 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.898370 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.898536 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.898662 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.898839 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.898965 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.899149 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.899283 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.899451 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.899584 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.899755 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.899881 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.900060 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.900194 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.900363 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.900488 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.900680 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.900810 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.901009 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.901145 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.901320 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.901458 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.901638 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.901762 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.901945 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.902070 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.902254 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.902380 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.902565 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.902691 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.902870 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.902994 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.903175 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.903300 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.903489 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.903616 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.903796 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.903920 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.904102 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.904233 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.904414 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.904549 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.904753 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.904880 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.905065 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.905191 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.905375 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.905516 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.905684 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.905816 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.905991 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.906115 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.906299 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.906436 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.906625 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.906758 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.906947 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.907071 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.907250 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.907382 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.907574 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.907700 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.907888 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.908016 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.908198 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.908323 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.908503 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.908628 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.908780 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 08 02:16:55.908903 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.909147 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.909457 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.909763 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.910070 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.910377 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.911702 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.912030 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.912348 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.912691 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.913002 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.913320 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.913648 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.913975 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.914287 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.914613 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.914940 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.915281 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.915598 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.915941 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.916267 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.916579 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.916915 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.917221 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:16:55.917463 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103871) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 08 02:16:56.069304 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: INFO dbcounter [None req-8ded7a98-38d6-4d56-858f-2af4f97b871f None None] Registered counter for database nova_api Aug 08 02:16:56.082194 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: INFO dbcounter [None req-5643cc36-c3bc-4971-9714-530528ee9bd6 None None] Registered counter for database nova_api Aug 08 02:16:56.090549 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG oslo_db.sqlalchemy.engines [None req-8ded7a98-38d6-4d56-858f-2af4f97b871f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103872) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 08 02:16:56.096983 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG dbcounter [-] [103872] Writer thread running {{(pid=103872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 08 02:16:56.103085 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG oslo_db.sqlalchemy.engines [None req-5643cc36-c3bc-4971-9714-530528ee9bd6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103871) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 08 02:16:56.109414 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG dbcounter [-] [103871] Writer thread running {{(pid=103871) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 08 02:16:56.152776 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.context [None req-8ded7a98-38d6-4d56-858f-2af4f97b871f None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),3f4309de-b8ab-46d7-9a8f-20127e13d27a(cell1) {{(pid=103872) load_cells /opt/stack/nova/nova/context.py:464}} Aug 08 02:16:56.156619 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG oslo_concurrency.lockutils [None req-8ded7a98-38d6-4d56-858f-2af4f97b871f None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103872) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:16:56.157119 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG oslo_concurrency.lockutils [None req-8ded7a98-38d6-4d56-858f-2af4f97b871f None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=103872) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:16:56.158706 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG oslo_concurrency.lockutils [None req-8ded7a98-38d6-4d56-858f-2af4f97b871f None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103872) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:16:56.159430 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG oslo_concurrency.lockutils [None req-8ded7a98-38d6-4d56-858f-2af4f97b871f None None] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103872) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:16:56.159879 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG oslo_concurrency.lockutils [None req-8ded7a98-38d6-4d56-858f-2af4f97b871f None None] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103872) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:16:56.162232 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG oslo_concurrency.lockutils [None req-8ded7a98-38d6-4d56-858f-2af4f97b871f None None] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103872) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:16:56.164218 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: INFO dbcounter [None req-8ded7a98-38d6-4d56-858f-2af4f97b871f None None] Registered counter for database nova_cell0 Aug 08 02:16:56.166425 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.context [None req-5643cc36-c3bc-4971-9714-530528ee9bd6 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),3f4309de-b8ab-46d7-9a8f-20127e13d27a(cell1) {{(pid=103871) load_cells /opt/stack/nova/nova/context.py:464}} Aug 08 02:16:56.169621 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: INFO dbcounter [None req-8ded7a98-38d6-4d56-858f-2af4f97b871f None None] Registered counter for database nova_cell1 Aug 08 02:16:56.170249 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG oslo_concurrency.lockutils [None req-5643cc36-c3bc-4971-9714-530528ee9bd6 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103871) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:16:56.170719 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG oslo_concurrency.lockutils [None req-5643cc36-c3bc-4971-9714-530528ee9bd6 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=103871) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:16:56.172363 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG oslo_concurrency.lockutils [None req-5643cc36-c3bc-4971-9714-530528ee9bd6 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103871) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:16:56.173012 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG oslo_concurrency.lockutils [None req-5643cc36-c3bc-4971-9714-530528ee9bd6 None None] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103871) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:16:56.173455 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG oslo_concurrency.lockutils [None req-5643cc36-c3bc-4971-9714-530528ee9bd6 None None] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103871) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:16:56.175462 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG oslo_concurrency.lockutils [None req-5643cc36-c3bc-4971-9714-530528ee9bd6 None None] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103871) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:16:56.177272 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG oslo_db.sqlalchemy.engines [None req-8ded7a98-38d6-4d56-858f-2af4f97b871f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103872) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 08 02:16:56.178770 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: INFO dbcounter [None req-5643cc36-c3bc-4971-9714-530528ee9bd6 None None] Registered counter for database nova_cell0 Aug 08 02:16:56.179592 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG oslo_db.sqlalchemy.engines [None req-8ded7a98-38d6-4d56-858f-2af4f97b871f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103872) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 08 02:16:56.184492 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: INFO dbcounter [None req-5643cc36-c3bc-4971-9714-530528ee9bd6 None None] Registered counter for database nova_cell1 Aug 08 02:16:56.185204 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG dbcounter [-] [103872] Writer thread running {{(pid=103872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 08 02:16:56.188060 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG dbcounter [-] [103872] Writer thread running {{(pid=103872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 08 02:16:56.191598 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG oslo_db.sqlalchemy.engines [None req-5643cc36-c3bc-4971-9714-530528ee9bd6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103871) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 08 02:16:56.194078 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG oslo_db.sqlalchemy.engines [None req-5643cc36-c3bc-4971-9714-530528ee9bd6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103871) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 08 02:16:56.200507 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG dbcounter [-] [103871] Writer thread running {{(pid=103871) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 08 02:16:56.201954 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG dbcounter [-] [103871] Writer thread running {{(pid=103871) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 08 02:16:56.333449 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: INFO dbcounter [None req-8ded7a98-38d6-4d56-858f-2af4f97b871f None None] Registered counter for database nova_cell0 Aug 08 02:16:56.342607 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG oslo_db.sqlalchemy.engines [None req-8ded7a98-38d6-4d56-858f-2af4f97b871f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103872) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 08 02:16:56.346833 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG dbcounter [-] [103872] Writer thread running {{(pid=103872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 08 02:16:56.353638 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: INFO dbcounter [None req-5643cc36-c3bc-4971-9714-530528ee9bd6 None None] Registered counter for database nova_cell0 Aug 08 02:16:56.364834 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG oslo_db.sqlalchemy.engines [None req-5643cc36-c3bc-4971-9714-530528ee9bd6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103871) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 08 02:16:56.369152 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG dbcounter [-] [103871] Writer thread running {{(pid=103871) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 08 02:16:56.765892 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: WARNING keystonemiddleware.auth_token [None req-8ded7a98-38d6-4d56-858f-2af4f97b871f None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Aug 08 02:16:56.773266 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: WARNING keystonemiddleware.auth_token [None req-8ded7a98-38d6-4d56-858f-2af4f97b871f None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Aug 08 02:16:56.799975 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: WARNING keystonemiddleware.auth_token [None req-5643cc36-c3bc-4971-9714-530528ee9bd6 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Aug 08 02:16:56.808681 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: WARNING keystonemiddleware.auth_token [None req-5643cc36-c3bc-4971-9714-530528ee9bd6 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Aug 08 02:16:57.112707 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: WARNING keystonemiddleware.auth_token [None req-8ded7a98-38d6-4d56-858f-2af4f97b871f None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Aug 08 02:16:57.119364 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: WARNING keystonemiddleware.auth_token [None req-8ded7a98-38d6-4d56-858f-2af4f97b871f None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Aug 08 02:16:57.124614 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x559cff3899c0 pid: 103872 (default app) Aug 08 02:16:57.126709 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi [None req-8ded7a98-38d6-4d56-858f-2af4f97b871f None None] Calling method '>' {{(pid=103872) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:16:57.127901 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: INFO nova.api.openstack.requestlog [None req-8ded7a98-38d6-4d56-858f-2af4f97b871f None None] 10.222.0.27 "GET /compute/v2.1/" status: 200 len: 388 microversion: 2.1 time: 0.001747 Aug 08 02:16:57.129480 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: [pid: 103872|app: 0|req: 1/1] 10.222.0.27 () {50 vars in 788 bytes} [Mon Aug 8 02:16:57 2022] GET /compute/v2.1/ => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:16:57.159000 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: WARNING keystonemiddleware.auth_token [None req-5643cc36-c3bc-4971-9714-530528ee9bd6 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Aug 08 02:16:57.166227 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: WARNING keystonemiddleware.auth_token [None req-5643cc36-c3bc-4971-9714-530528ee9bd6 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Aug 08 02:16:57.171607 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x559cff3899c0 pid: 103871 (default app) Aug 08 02:18:16.320377 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi [None req-8ded7a98-38d6-4d56-858f-2af4f97b871f None None] Calling method '>' {{(pid=103872) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:18:16.321814 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: INFO nova.api.openstack.requestlog [None req-8ded7a98-38d6-4d56-858f-2af4f97b871f None None] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001908 Aug 08 02:18:16.322466 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: [pid: 103872|app: 0|req: 2/2] 10.222.0.27 () {54 vars in 931 bytes} [Mon Aug 8 02:18:16 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:18:16.339689 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG dbcounter [-] [103871] Writing DB stats nova_api:SELECT=3 {{(pid=103871) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:16.341076 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG dbcounter [-] [103871] Writing DB stats nova_cell0:SELECT=3 {{(pid=103871) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:16.341912 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG dbcounter [-] [103871] Writing DB stats nova_cell1:SELECT=3 {{(pid=103871) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:16.342765 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG dbcounter [-] [103871] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=103871) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:17.262825 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi [None req-d2232ca2-ce94-4a53-ac64-34938aafa7c5 admin admin] Calling method '>' {{(pid=103871) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:18:17.302580 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: INFO nova.api.openstack.requestlog [None req-d2232ca2-ce94-4a53-ac64-34938aafa7c5 admin admin] 10.222.0.27 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 15 microversion: 2.61 time: 0.972969 Aug 08 02:18:17.303351 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: [pid: 103871|app: 0|req: 1/3] 10.222.0.27 () {60 vars in 1292 bytes} [Mon Aug 8 02:18:16 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 15 bytes in 975 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:18:19.530409 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi [None req-8ded7a98-38d6-4d56-858f-2af4f97b871f None None] Calling method '>' {{(pid=103872) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:18:19.534145 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: INFO nova.api.openstack.requestlog [None req-8ded7a98-38d6-4d56-858f-2af4f97b871f None None] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001496 Aug 08 02:18:19.534145 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: [pid: 103872|app: 0|req: 3/4] 10.222.0.27 () {54 vars in 931 bytes} [Mon Aug 8 02:18:19 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:18:20.030084 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi [None req-5121bc5e-9086-4430-a88b-0bcf1c097506 admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 1, "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "disk": 1, "os-flavor-access:is_public": true, "ram": 256, "rxtx_factor": 1.0, "id": "c1", "name": "cirros256"}} {{(pid=103871) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:18:20.050464 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:18:20.050464 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: warnings.warn(msg) Aug 08 02:18:20.135663 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: INFO nova.api.openstack.requestlog [None req-5121bc5e-9086-4430-a88b-0bcf1c097506 admin admin] 10.222.0.27 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.61 time: 0.596032 Aug 08 02:18:20.136946 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: [pid: 103871|app: 0|req: 2/5] 10.222.0.27 () {64 vars in 1283 bytes} [Mon Aug 8 02:18:19 2022] POST /compute/v2.1/flavors => generated 407 bytes in 597 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 08 02:18:20.351310 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi [None req-57b7b8b7-470b-403d-8b23-975c4a18a343 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103872) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:18:20.355632 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-extra-specs:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:18:20.355632 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: warnings.warn(msg) Aug 08 02:18:20.375062 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG dbcounter [-] [103872] Writing DB stats nova_api:SELECT=3 {{(pid=103872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:20.375949 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG dbcounter [-] [103872] Writing DB stats nova_cell0:SELECT=3 {{(pid=103872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:20.376784 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG dbcounter [-] [103872] Writing DB stats nova_cell1:SELECT=3 {{(pid=103872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:20.377788 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG dbcounter [-] [103872] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=103872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:20.478285 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: INFO nova.api.openstack.requestlog [None req-57b7b8b7-470b-403d-8b23-975c4a18a343 admin admin] 10.222.0.27 "POST /compute/v2.1/flavors/c1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.332917 Aug 08 02:18:20.487014 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: [pid: 103872|app: 0|req: 4/6] 10.222.0.27 () {64 vars in 1336 bytes} [Mon Aug 8 02:18:20 2022] POST /compute/v2.1/flavors/c1/os-extra_specs => generated 43 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:18:22.718409 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi [None req-5121bc5e-9086-4430-a88b-0bcf1c097506 admin admin] Calling method '>' {{(pid=103871) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:18:22.719497 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: INFO nova.api.openstack.requestlog [None req-5121bc5e-9086-4430-a88b-0bcf1c097506 admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001564 Aug 08 02:18:22.720045 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: [pid: 103871|app: 0|req: 3/7] 10.222.0.27 () {54 vars in 931 bytes} [Mon Aug 8 02:18:22 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:18:23.367809 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi [None req-ef2fa372-752a-48b5-bca4-cd4991a133b0 admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "d1", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "disk": 5, "vcpus": 1, "name": "ds512M", "ram": 512, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103872) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:18:23.390108 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:18:23.390108 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: warnings.warn(msg) Aug 08 02:18:23.480612 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: INFO nova.api.openstack.requestlog [None req-ef2fa372-752a-48b5-bca4-cd4991a133b0 admin admin] 10.222.0.27 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.753410 Aug 08 02:18:23.481550 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: [pid: 103872|app: 0|req: 5/8] 10.222.0.27 () {64 vars in 1283 bytes} [Mon Aug 8 02:18:22 2022] POST /compute/v2.1/flavors => generated 404 bytes in 755 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 08 02:18:23.493649 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi [None req-8d7a63c5-6fe0-4ff1-9a54-aaf4c635dc26 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103871) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:18:23.496498 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-extra-specs:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:18:23.496498 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: warnings.warn(msg) Aug 08 02:18:23.608854 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: INFO nova.api.openstack.requestlog [None req-8d7a63c5-6fe0-4ff1-9a54-aaf4c635dc26 admin admin] 10.222.0.27 "POST /compute/v2.1/flavors/d1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.118965 Aug 08 02:18:23.616877 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: [pid: 103871|app: 0|req: 4/9] 10.222.0.27 () {64 vars in 1336 bytes} [Mon Aug 8 02:18:23 2022] POST /compute/v2.1/flavors/d1/os-extra_specs => generated 43 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:18:25.837422 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi [None req-ef2fa372-752a-48b5-bca4-cd4991a133b0 admin admin] Calling method '>' {{(pid=103872) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:18:25.838434 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: INFO nova.api.openstack.requestlog [None req-ef2fa372-752a-48b5-bca4-cd4991a133b0 admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001511 Aug 08 02:18:25.840970 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: [pid: 103872|app: 0|req: 6/10] 10.222.0.27 () {54 vars in 931 bytes} [Mon Aug 8 02:18:25 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:18:26.234887 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi [None req-b99ac850-1c8d-457f-803e-255046415e30 admin admin] Action: 'create', calling method: >, body: {"flavor": {"OS-FLV-EXT-DATA:ephemeral": 0, "ram": 1024, "id": "d2", "swap": 0, "rxtx_factor": 1.0, "name": "ds1G", "os-flavor-access:is_public": true, "disk": 10, "vcpus": 1}} {{(pid=103871) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:18:26.306944 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: INFO nova.api.openstack.requestlog [None req-b99ac850-1c8d-457f-803e-255046415e30 admin admin] 10.222.0.27 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.462208 Aug 08 02:18:26.307452 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: [pid: 103871|app: 0|req: 5/11] 10.222.0.27 () {64 vars in 1283 bytes} [Mon Aug 8 02:18:25 2022] POST /compute/v2.1/flavors => generated 404 bytes in 463 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 08 02:18:26.319515 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi [None req-0a9d8f4b-76be-420f-9b2d-8d182e749bba admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103872) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:18:26.401960 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: INFO nova.api.openstack.requestlog [None req-0a9d8f4b-76be-420f-9b2d-8d182e749bba admin admin] 10.222.0.27 "POST /compute/v2.1/flavors/d2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.086069 Aug 08 02:18:26.402728 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: [pid: 103872|app: 0|req: 7/12] 10.222.0.27 () {64 vars in 1336 bytes} [Mon Aug 8 02:18:26 2022] POST /compute/v2.1/flavors/d2/os-extra_specs => generated 43 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:18:28.662999 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi [None req-b99ac850-1c8d-457f-803e-255046415e30 admin admin] Calling method '>' {{(pid=103871) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:18:28.663985 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: INFO nova.api.openstack.requestlog [None req-b99ac850-1c8d-457f-803e-255046415e30 admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001471 Aug 08 02:18:28.664394 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: [pid: 103871|app: 0|req: 6/13] 10.222.0.27 () {54 vars in 931 bytes} [Mon Aug 8 02:18:28 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 08 02:18:29.110531 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi [None req-9ab1682e-84dc-4e9b-9e04-b78f4f90aacf admin admin] Action: 'create', calling method: >, body: {"flavor": {"rxtx_factor": 1.0, "id": "d3", "swap": 0, "vcpus": 2, "OS-FLV-EXT-DATA:ephemeral": 0, "ram": 2048, "os-flavor-access:is_public": true, "name": "ds2G", "disk": 10}} {{(pid=103872) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:18:29.267337 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: INFO nova.api.openstack.requestlog [None req-9ab1682e-84dc-4e9b-9e04-b78f4f90aacf admin admin] 10.222.0.27 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.597080 Aug 08 02:18:29.269000 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: [pid: 103872|app: 0|req: 8/14] 10.222.0.27 () {64 vars in 1283 bytes} [Mon Aug 8 02:18:28 2022] POST /compute/v2.1/flavors => generated 404 bytes in 599 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 08 02:18:29.280919 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi [None req-c4723387-f0d9-4500-98b5-8767df1debe1 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103871) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:18:29.402957 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: INFO nova.api.openstack.requestlog [None req-c4723387-f0d9-4500-98b5-8767df1debe1 admin admin] 10.222.0.27 "POST /compute/v2.1/flavors/d3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.125801 Aug 08 02:18:29.403887 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: [pid: 103871|app: 0|req: 7/15] 10.222.0.27 () {64 vars in 1336 bytes} [Mon Aug 8 02:18:29 2022] POST /compute/v2.1/flavors/d3/os-extra_specs => generated 43 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:18:31.618084 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi [None req-9ab1682e-84dc-4e9b-9e04-b78f4f90aacf admin admin] Calling method '>' {{(pid=103872) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:18:31.619103 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: INFO nova.api.openstack.requestlog [None req-9ab1682e-84dc-4e9b-9e04-b78f4f90aacf admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001463 Aug 08 02:18:31.619507 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: [pid: 103872|app: 0|req: 9/16] 10.222.0.27 () {54 vars in 931 bytes} [Mon Aug 8 02:18:31 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:18:32.044517 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi [None req-a01c410e-5db8-4508-b464-39c3f41f7c2f admin admin] Action: 'create', calling method: >, body: {"flavor": {"OS-FLV-EXT-DATA:ephemeral": 0, "id": "d4", "ram": 4096, "name": "ds4G", "disk": 20, "vcpus": 4, "swap": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103871) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:18:32.124734 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: INFO nova.api.openstack.requestlog [None req-a01c410e-5db8-4508-b464-39c3f41f7c2f admin admin] 10.222.0.27 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.499217 Aug 08 02:18:32.125440 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: [pid: 103871|app: 0|req: 8/17] 10.222.0.27 () {64 vars in 1283 bytes} [Mon Aug 8 02:18:31 2022] POST /compute/v2.1/flavors => generated 404 bytes in 501 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 08 02:18:32.138456 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi [None req-6397eb0e-0308-4fdc-be10-fe1bc3428cd2 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103872) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:18:32.258056 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: INFO nova.api.openstack.requestlog [None req-6397eb0e-0308-4fdc-be10-fe1bc3428cd2 admin admin] 10.222.0.27 "POST /compute/v2.1/flavors/d4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.123553 Aug 08 02:18:32.258762 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: [pid: 103872|app: 0|req: 10/18] 10.222.0.27 () {64 vars in 1336 bytes} [Mon Aug 8 02:18:32 2022] POST /compute/v2.1/flavors/d4/os-extra_specs => generated 43 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:18:34.483863 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi [None req-a01c410e-5db8-4508-b464-39c3f41f7c2f admin admin] Calling method '>' {{(pid=103871) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:18:34.485236 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: INFO nova.api.openstack.requestlog [None req-a01c410e-5db8-4508-b464-39c3f41f7c2f admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001962 Aug 08 02:18:34.485728 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: [pid: 103871|app: 0|req: 9/19] 10.222.0.27 () {54 vars in 931 bytes} [Mon Aug 8 02:18:34 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:18:34.933095 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi [None req-d3cef145-9040-464c-81ac-e51944e627e1 admin admin] Calling method '>' {{(pid=103872) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:18:34.960311 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: INFO nova.api.openstack.requestlog [None req-d3cef145-9040-464c-81ac-e51944e627e1 admin admin] 10.222.0.27 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 2106 microversion: 2.61 time: 0.467610 Aug 08 02:18:34.960881 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: [pid: 103872|app: 0|req: 11/20] 10.222.0.27 () {60 vars in 1292 bytes} [Mon Aug 8 02:18:34 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 2106 bytes in 469 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:18:37.188634 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi [None req-a01c410e-5db8-4508-b464-39c3f41f7c2f admin admin] Calling method '>' {{(pid=103871) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:18:37.191289 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: INFO nova.api.openstack.requestlog [None req-a01c410e-5db8-4508-b464-39c3f41f7c2f admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.003252 Aug 08 02:18:37.191774 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: [pid: 103871|app: 0|req: 10/21] 10.222.0.27 () {54 vars in 931 bytes} [Mon Aug 8 02:18:37 2022] GET /compute/v2.1 => generated 388 bytes in 5 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:18:37.594372 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi [None req-1601c9e6-bd26-4444-be3a-727b5352ff08 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.tiny", "os-flavor-access:is_public": true, "ram": 512, "id": "1", "vcpus": 1, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "swap": 0, "disk": 1}} {{(pid=103872) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:18:37.806455 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: INFO nova.api.openstack.requestlog [None req-1601c9e6-bd26-4444-be3a-727b5352ff08 admin admin] 10.222.0.27 "POST /compute/v2.1/flavors" status: 200 len: 402 microversion: 2.61 time: 0.606760 Aug 08 02:18:37.807202 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: [pid: 103872|app: 0|req: 12/22] 10.222.0.27 () {64 vars in 1283 bytes} [Mon Aug 8 02:18:37 2022] POST /compute/v2.1/flavors => generated 402 bytes in 608 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 08 02:18:37.821948 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi [None req-6a63ba97-8199-4fc8-9c3e-29ae18a15ee6 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103871) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:18:37.980367 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: INFO nova.api.openstack.requestlog [None req-6a63ba97-8199-4fc8-9c3e-29ae18a15ee6 admin admin] 10.222.0.27 "POST /compute/v2.1/flavors/1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.163472 Aug 08 02:18:37.980991 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: [pid: 103871|app: 0|req: 11/23] 10.222.0.27 () {64 vars in 1333 bytes} [Mon Aug 8 02:18:37 2022] POST /compute/v2.1/flavors/1/os-extra_specs => generated 43 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:18:40.339307 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi [None req-1601c9e6-bd26-4444-be3a-727b5352ff08 admin admin] Calling method '>' {{(pid=103872) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:18:40.340348 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: INFO nova.api.openstack.requestlog [None req-1601c9e6-bd26-4444-be3a-727b5352ff08 admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001458 Aug 08 02:18:40.340773 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: [pid: 103872|app: 0|req: 13/24] 10.222.0.27 () {54 vars in 931 bytes} [Mon Aug 8 02:18:40 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:18:40.864080 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi [None req-f2c2b8bc-ddf5-4263-a87f-8d9d9410a560 admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 1, "ram": 2048, "disk": 20, "rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true, "name": "m1.small", "swap": 0, "id": "2"}} {{(pid=103871) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:18:40.946104 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: INFO nova.api.openstack.requestlog [None req-f2c2b8bc-ddf5-4263-a87f-8d9d9410a560 admin admin] 10.222.0.27 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.599210 Aug 08 02:18:40.948805 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: [pid: 103871|app: 0|req: 12/25] 10.222.0.27 () {64 vars in 1283 bytes} [Mon Aug 8 02:18:40 2022] POST /compute/v2.1/flavors => generated 405 bytes in 600 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 08 02:18:40.959097 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi [None req-ee642c12-7749-4dec-8af9-481fa345630f admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103872) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:18:41.046285 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: INFO nova.api.openstack.requestlog [None req-ee642c12-7749-4dec-8af9-481fa345630f admin admin] 10.222.0.27 "POST /compute/v2.1/flavors/2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.091066 Aug 08 02:18:41.046868 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: [pid: 103872|app: 0|req: 14/26] 10.222.0.27 () {64 vars in 1333 bytes} [Mon Aug 8 02:18:40 2022] POST /compute/v2.1/flavors/2/os-extra_specs => generated 43 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:18:43.264288 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi [None req-f2c2b8bc-ddf5-4263-a87f-8d9d9410a560 admin admin] Calling method '>' {{(pid=103871) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:18:43.265401 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: INFO nova.api.openstack.requestlog [None req-f2c2b8bc-ddf5-4263-a87f-8d9d9410a560 admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002039 Aug 08 02:18:43.265821 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: [pid: 103871|app: 0|req: 13/27] 10.222.0.27 () {54 vars in 931 bytes} [Mon Aug 8 02:18:43 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:18:43.655118 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi [None req-e26a314b-84ac-48ee-bd38-98b2668cc27f admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 40, "os-flavor-access:is_public": true, "name": "m1.medium", "ram": 4096, "id": "3", "swap": 0, "vcpus": 2, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0}} {{(pid=103872) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:18:43.734849 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: INFO nova.api.openstack.requestlog [None req-e26a314b-84ac-48ee-bd38-98b2668cc27f admin admin] 10.222.0.27 "POST /compute/v2.1/flavors" status: 200 len: 406 microversion: 2.61 time: 0.461098 Aug 08 02:18:43.735416 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: [pid: 103872|app: 0|req: 15/28] 10.222.0.27 () {64 vars in 1283 bytes} [Mon Aug 8 02:18:43 2022] POST /compute/v2.1/flavors => generated 406 bytes in 462 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 08 02:18:43.748052 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi [None req-e51dc241-464b-4fbf-901b-f67e893aac26 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103871) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:18:43.871287 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: INFO nova.api.openstack.requestlog [None req-e51dc241-464b-4fbf-901b-f67e893aac26 admin admin] 10.222.0.27 "POST /compute/v2.1/flavors/3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.126931 Aug 08 02:18:43.871853 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: [pid: 103871|app: 0|req: 14/29] 10.222.0.27 () {64 vars in 1333 bytes} [Mon Aug 8 02:18:43 2022] POST /compute/v2.1/flavors/3/os-extra_specs => generated 43 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:18:46.090440 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi [None req-e26a314b-84ac-48ee-bd38-98b2668cc27f admin admin] Calling method '>' {{(pid=103872) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:18:46.092083 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: INFO nova.api.openstack.requestlog [None req-e26a314b-84ac-48ee-bd38-98b2668cc27f admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002073 Aug 08 02:18:46.092486 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: [pid: 103872|app: 0|req: 16/30] 10.222.0.27 () {54 vars in 931 bytes} [Mon Aug 8 02:18:46 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:18:46.559280 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi [None req-14dc467b-c92d-4d9d-9eaf-d152b7fe5488 admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 8192, "os-flavor-access:is_public": true, "name": "m1.large", "id": "4", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 4, "disk": 80, "rxtx_factor": 1.0}} {{(pid=103871) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:18:46.566948 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG dbcounter [-] [103871] Writing DB stats nova_api:SELECT=39,nova_api:INSERT=8,nova_api:SAVEPOINT=4,nova_api:RELEASE=4 {{(pid=103871) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:46.659855 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: INFO nova.api.openstack.requestlog [None req-14dc467b-c92d-4d9d-9eaf-d152b7fe5488 admin admin] 10.222.0.27 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.560390 Aug 08 02:18:46.660432 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: [pid: 103871|app: 0|req: 15/31] 10.222.0.27 () {64 vars in 1283 bytes} [Mon Aug 8 02:18:46 2022] POST /compute/v2.1/flavors => generated 405 bytes in 562 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 08 02:18:46.674259 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi [None req-c080d67f-4762-4cfd-828d-a34ed0812865 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103872) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:18:46.761003 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: INFO nova.api.openstack.requestlog [None req-c080d67f-4762-4cfd-828d-a34ed0812865 admin admin] 10.222.0.27 "POST /compute/v2.1/flavors/4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.090644 Aug 08 02:18:46.761756 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: [pid: 103872|app: 0|req: 17/32] 10.222.0.27 () {64 vars in 1333 bytes} [Mon Aug 8 02:18:46 2022] POST /compute/v2.1/flavors/4/os-extra_specs => generated 43 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:18:49.068648 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi [None req-14dc467b-c92d-4d9d-9eaf-d152b7fe5488 admin admin] Calling method '>' {{(pid=103871) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:18:49.070393 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: INFO nova.api.openstack.requestlog [None req-14dc467b-c92d-4d9d-9eaf-d152b7fe5488 admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002175 Aug 08 02:18:49.070981 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: [pid: 103871|app: 0|req: 16/33] 10.222.0.27 () {54 vars in 931 bytes} [Mon Aug 8 02:18:49 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:18:49.516827 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi [None req-6ffc9a0a-54be-422e-85bb-de90efbeb3bc admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.xlarge", "id": "5", "os-flavor-access:is_public": true, "disk": 160, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "swap": 0, "vcpus": 8, "ram": 16384}} {{(pid=103872) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:18:49.605241 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: INFO nova.api.openstack.requestlog [None req-6ffc9a0a-54be-422e-85bb-de90efbeb3bc admin admin] 10.222.0.27 "POST /compute/v2.1/flavors" status: 200 len: 408 microversion: 2.61 time: 0.528018 Aug 08 02:18:49.613112 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: [pid: 103872|app: 0|req: 18/34] 10.222.0.27 () {64 vars in 1283 bytes} [Mon Aug 8 02:18:49 2022] POST /compute/v2.1/flavors => generated 408 bytes in 529 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 08 02:18:49.623772 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi [None req-89fe67a4-5e61-4660-b4fd-78337a0975d8 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103871) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:18:49.710990 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: INFO nova.api.openstack.requestlog [None req-89fe67a4-5e61-4660-b4fd-78337a0975d8 admin admin] 10.222.0.27 "POST /compute/v2.1/flavors/5/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.090757 Aug 08 02:18:49.712645 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: [pid: 103871|app: 0|req: 17/35] 10.222.0.27 () {64 vars in 1333 bytes} [Mon Aug 8 02:18:49 2022] POST /compute/v2.1/flavors/5/os-extra_specs => generated 43 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:19:10.247361 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi [None req-6ffc9a0a-54be-422e-85bb-de90efbeb3bc admin admin] Calling method '>' {{(pid=103872) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:19:10.248392 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: INFO nova.api.openstack.requestlog [None req-6ffc9a0a-54be-422e-85bb-de90efbeb3bc admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001462 Aug 08 02:19:10.248816 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: [pid: 103872|app: 0|req: 19/36] 10.222.0.27 () {54 vars in 931 bytes} [Mon Aug 8 02:19:10 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:19:10.635279 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi [None req-c589204d-0a82-4c7c-8ae4-5e614225e5c3 admin admin] Calling method '>' {{(pid=103871) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:19:10.653476 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: INFO nova.api.openstack.requestlog [None req-c589204d-0a82-4c7c-8ae4-5e614225e5c3 admin admin] 10.222.0.27 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 4202 microversion: 2.61 time: 0.398652 Aug 08 02:19:10.654013 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: [pid: 103871|app: 0|req: 18/37] 10.222.0.27 () {60 vars in 1292 bytes} [Mon Aug 8 02:19:10 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 4202 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 08 02:19:11.713517 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi [None req-6ffc9a0a-54be-422e-85bb-de90efbeb3bc admin admin] Calling method '>' {{(pid=103872) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:19:11.714886 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: INFO nova.api.openstack.requestlog [None req-6ffc9a0a-54be-422e-85bb-de90efbeb3bc admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001757 Aug 08 02:19:11.715405 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: [pid: 103872|app: 0|req: 20/38] 10.222.0.27 () {56 vars in 1074 bytes} [Mon Aug 8 02:19:11 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:19:12.232162 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi [None req-3be0fd03-3b5a-4d3f-9550-2ab369556a50 admin admin] Calling method '>' {{(pid=103871) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:19:12.237808 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: INFO nova.api.openstack.wsgi [None req-3be0fd03-3b5a-4d3f-9550-2ab369556a50 admin admin] HTTP exception thrown: Flavor 42 could not be found. Aug 08 02:19:12.241067 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi [None req-3be0fd03-3b5a-4d3f-9550-2ab369556a50 admin admin] Request method failure captured: Aug 08 02:19:12.241067 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Aug 08 02:19:12.241067 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: Accept: application/json Aug 08 02:19:12.241067 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: Accept-Encoding: gzip, deflate Aug 08 02:19:12.241067 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: Connection: keep-alive Aug 08 02:19:12.241067 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: Content-Length: 0 Aug 08 02:19:12.241067 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: Host: 10.222.0.27 Aug 08 02:19:12.241067 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: Openstack-Api-Version: compute 2.90 Aug 08 02:19:12.241067 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: Openstack-System-Scope: None Aug 08 02:19:12.241067 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: User-Agent: python-novaclient Aug 08 02:19:12.241067 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-Auth-Token: gAAAAABi8HKfOuuCf6AwrKp-e-ak1UXrmIRy1bfO7Ejxp12-94w6MTxwUTX7jkjBOulFYEePzFOLxE6votYj_jBrp0TPt-If4p8mTYpA0gAt4SQ_uJjlzMHFIrY090vVZ_KCrg5nsyXov55AkMh95zBlYwcHzvWQMSwKmESuU5mF5Ug-wVSEIqg Aug 08 02:19:12.241067 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-Domain-Id: None Aug 08 02:19:12.241067 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-Domain-Name: None Aug 08 02:19:12.241067 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-Identity-Status: Confirmed Aug 08 02:19:12.241067 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-Is-Admin-Project: True Aug 08 02:19:12.241067 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-Openstack-Nova-Api-Version: 2.90 Aug 08 02:19:12.241067 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-Project-Domain-Id: default Aug 08 02:19:12.241067 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-Project-Domain-Name: Default Aug 08 02:19:12.241067 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-Project-Id: 5f307f793c6c464eafa52c1ca17d9bc6 Aug 08 02:19:12.241067 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-Project-Name: admin Aug 08 02:19:12.241067 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-Role: reader,member,admin Aug 08 02:19:12.241067 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-Roles: reader,member,admin Aug 08 02:19:12.242710 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/5f307f793c6c464eafa52c1ca17d9bc6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/5f307f793c6c464eafa52c1ca17d9bc6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_5f307f793c6c464eafa52c1ca17d9bc6", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/5f307f793c6c464eafa52c1ca17d9bc6"}]}] Aug 08 02:19:12.242710 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-Tenant: admin Aug 08 02:19:12.242710 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-Tenant-Id: 5f307f793c6c464eafa52c1ca17d9bc6 Aug 08 02:19:12.242710 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-Tenant-Name: admin Aug 08 02:19:12.242710 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-User: admin Aug 08 02:19:12.242710 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-User-Domain-Id: default Aug 08 02:19:12.242710 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-User-Domain-Name: Default Aug 08 02:19:12.242710 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-User-Id: 979e7eb5d5594f0ea4d2839dfb577133 Aug 08 02:19:12.242710 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-User-Name: admin Aug 08 02:19:12.242710 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: method: > Aug 08 02:19:12.242710 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: exception: Flavor 42 could not be found. Aug 08 02:19:12.242710 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: action_args: {'id': '42'} Aug 08 02:19:12.242710 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: {{(pid=103871) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:19:12.242710 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:19:12.243710 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Aug 08 02:19:12.243710 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Aug 08 02:19:12.243710 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Aug 08 02:19:12.243710 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Aug 08 02:19:12.243710 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Aug 08 02:19:12.243710 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Aug 08 02:19:12.243710 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Aug 08 02:19:12.243710 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Aug 08 02:19:12.243710 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Aug 08 02:19:12.243710 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:19:12.243710 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Aug 08 02:19:12.243710 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Aug 08 02:19:12.243710 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Aug 08 02:19:12.243710 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi Aug 08 02:19:12.243710 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:19:12.243710 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi Aug 08 02:19:12.243710 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:19:12.243710 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:19:12.243710 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:19:12.245086 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:19:12.245086 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:19:12.245086 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:19:12.245086 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:19:12.245086 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Aug 08 02:19:12.245086 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:19:12.245086 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Aug 08 02:19:12.245086 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi Aug 08 02:19:12.245086 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:19:12.245086 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi Aug 08 02:19:12.245086 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:19:12.245086 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:19:12.245086 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:19:12.245086 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:19:12.245086 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:19:12.245086 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Aug 08 02:19:12.245086 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi Aug 08 02:19:12.245086 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi [None req-3be0fd03-3b5a-4d3f-9550-2ab369556a50 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=103871) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:19:12.246353 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: INFO nova.api.openstack.requestlog [None req-3be0fd03-3b5a-4d3f-9550-2ab369556a50 admin admin] 10.222.0.27 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.90 time: 0.387415 Aug 08 02:19:12.246353 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: [pid: 103871|app: 0|req: 19/39] 10.222.0.27 () {60 vars in 1190 bytes} [Mon Aug 8 02:19:11 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 388 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Aug 08 02:19:12.252183 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi [None req-5219c71b-14c6-4f6c-b855-e4390c9de721 admin admin] Calling method '>' {{(pid=103872) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:19:12.256842 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG dbcounter [-] [103872] Writing DB stats nova_api:SELECT=47,nova_api:SAVEPOINT=5,nova_api:INSERT=10,nova_api:RELEASE=5 {{(pid=103872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:12.275893 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: INFO nova.api.openstack.requestlog [None req-5219c71b-14c6-4f6c-b855-e4390c9de721 admin admin] 10.222.0.27 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.027241 Aug 08 02:19:12.276427 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: [pid: 103872|app: 0|req: 21/40] 10.222.0.27 () {60 vars in 1210 bytes} [Mon Aug 8 02:19:12 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:19:12.286319 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi [None req-93029a65-99e4-4581-9c5a-ba7ddcba1b95 admin admin] Calling method '>' {{(pid=103871) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:19:12.310435 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: INFO nova.api.openstack.requestlog [None req-93029a65-99e4-4581-9c5a-ba7ddcba1b95 admin admin] 10.222.0.27 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.027727 Aug 08 02:19:12.310996 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: [pid: 103871|app: 0|req: 20/41] 10.222.0.27 () {60 vars in 1210 bytes} [Mon Aug 8 02:19:12 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:19:12.321323 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi [None req-a335c8ed-c54e-4e22-8cfd-04bba63522a3 admin admin] Calling method '>' {{(pid=103872) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:19:12.327370 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: INFO nova.api.openstack.wsgi [None req-a335c8ed-c54e-4e22-8cfd-04bba63522a3 admin admin] HTTP exception thrown: Flavor 42 could not be found. Aug 08 02:19:12.331320 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi [None req-a335c8ed-c54e-4e22-8cfd-04bba63522a3 admin admin] Request method failure captured: Aug 08 02:19:12.331320 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Aug 08 02:19:12.331320 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: Accept: application/json Aug 08 02:19:12.331320 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: Accept-Encoding: gzip, deflate Aug 08 02:19:12.331320 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: Connection: keep-alive Aug 08 02:19:12.331320 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: Content-Length: 0 Aug 08 02:19:12.331320 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: Host: 10.222.0.27 Aug 08 02:19:12.331320 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: Openstack-Api-Version: compute 2.90 Aug 08 02:19:12.331320 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: Openstack-System-Scope: None Aug 08 02:19:12.331320 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: User-Agent: python-novaclient Aug 08 02:19:12.331320 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-Auth-Token: gAAAAABi8HKfOuuCf6AwrKp-e-ak1UXrmIRy1bfO7Ejxp12-94w6MTxwUTX7jkjBOulFYEePzFOLxE6votYj_jBrp0TPt-If4p8mTYpA0gAt4SQ_uJjlzMHFIrY090vVZ_KCrg5nsyXov55AkMh95zBlYwcHzvWQMSwKmESuU5mF5Ug-wVSEIqg Aug 08 02:19:12.331320 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-Domain-Id: None Aug 08 02:19:12.331320 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-Domain-Name: None Aug 08 02:19:12.331320 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-Identity-Status: Confirmed Aug 08 02:19:12.331320 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-Is-Admin-Project: True Aug 08 02:19:12.331320 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-Openstack-Nova-Api-Version: 2.90 Aug 08 02:19:12.331320 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-Project-Domain-Id: default Aug 08 02:19:12.331320 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-Project-Domain-Name: Default Aug 08 02:19:12.331320 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-Project-Id: 5f307f793c6c464eafa52c1ca17d9bc6 Aug 08 02:19:12.331320 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-Project-Name: admin Aug 08 02:19:12.331320 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-Role: reader,member,admin Aug 08 02:19:12.331320 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-Roles: reader,member,admin Aug 08 02:19:12.333297 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/5f307f793c6c464eafa52c1ca17d9bc6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/5f307f793c6c464eafa52c1ca17d9bc6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_5f307f793c6c464eafa52c1ca17d9bc6", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/5f307f793c6c464eafa52c1ca17d9bc6"}]}] Aug 08 02:19:12.333297 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-Tenant: admin Aug 08 02:19:12.333297 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-Tenant-Id: 5f307f793c6c464eafa52c1ca17d9bc6 Aug 08 02:19:12.333297 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-Tenant-Name: admin Aug 08 02:19:12.333297 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-User: admin Aug 08 02:19:12.333297 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-User-Domain-Id: default Aug 08 02:19:12.333297 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-User-Domain-Name: Default Aug 08 02:19:12.333297 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-User-Id: 979e7eb5d5594f0ea4d2839dfb577133 Aug 08 02:19:12.333297 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-User-Name: admin Aug 08 02:19:12.333297 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: method: > Aug 08 02:19:12.333297 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: exception: Flavor 42 could not be found. Aug 08 02:19:12.333297 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: action_args: {'id': '42'} Aug 08 02:19:12.333297 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: {{(pid=103872) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:19:12.333297 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:19:12.334109 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Aug 08 02:19:12.334109 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Aug 08 02:19:12.334109 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Aug 08 02:19:12.334109 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Aug 08 02:19:12.334109 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Aug 08 02:19:12.334109 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Aug 08 02:19:12.334109 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Aug 08 02:19:12.334109 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Aug 08 02:19:12.334109 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Aug 08 02:19:12.334109 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:19:12.334109 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Aug 08 02:19:12.334109 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Aug 08 02:19:12.334109 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Aug 08 02:19:12.334109 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi Aug 08 02:19:12.334109 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:19:12.334109 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi Aug 08 02:19:12.334109 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:19:12.334109 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:19:12.334109 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:19:12.335161 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:19:12.335161 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:19:12.335161 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:19:12.335161 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:19:12.335161 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Aug 08 02:19:12.335161 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:19:12.335161 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Aug 08 02:19:12.335161 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi Aug 08 02:19:12.335161 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:19:12.335161 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi Aug 08 02:19:12.335161 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:19:12.335161 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:19:12.335161 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:19:12.335161 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:19:12.335161 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:19:12.335161 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Aug 08 02:19:12.335161 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi Aug 08 02:19:12.335161 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi [None req-a335c8ed-c54e-4e22-8cfd-04bba63522a3 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=103872) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:19:12.336216 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: INFO nova.api.openstack.requestlog [None req-a335c8ed-c54e-4e22-8cfd-04bba63522a3 admin admin] 10.222.0.27 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.90 time: 0.015216 Aug 08 02:19:12.336216 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: [pid: 103872|app: 0|req: 22/42] 10.222.0.27 () {60 vars in 1190 bytes} [Mon Aug 8 02:19:12 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 16 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Aug 08 02:19:13.312432 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi [None req-93029a65-99e4-4581-9c5a-ba7ddcba1b95 admin admin] Calling method '>' {{(pid=103871) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:19:13.313499 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: INFO nova.api.openstack.requestlog [None req-93029a65-99e4-4581-9c5a-ba7ddcba1b95 admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001516 Aug 08 02:19:13.313876 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: [pid: 103871|app: 0|req: 21/43] 10.222.0.27 () {56 vars in 1074 bytes} [Mon Aug 8 02:19:13 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:19:13.829229 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi [None req-ef0e1299-b508-4af2-8745-b539ed1496af admin admin] Calling method '>' {{(pid=103872) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:19:13.835267 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: INFO nova.api.openstack.wsgi [None req-ef0e1299-b508-4af2-8745-b539ed1496af admin admin] HTTP exception thrown: Flavor 84 could not be found. Aug 08 02:19:13.836817 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi [None req-ef0e1299-b508-4af2-8745-b539ed1496af admin admin] Request method failure captured: Aug 08 02:19:13.836817 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Aug 08 02:19:13.836817 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: Accept: application/json Aug 08 02:19:13.836817 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: Accept-Encoding: gzip, deflate Aug 08 02:19:13.836817 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: Connection: keep-alive Aug 08 02:19:13.836817 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: Content-Length: 0 Aug 08 02:19:13.836817 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: Host: 10.222.0.27 Aug 08 02:19:13.836817 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: Openstack-Api-Version: compute 2.90 Aug 08 02:19:13.836817 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: Openstack-System-Scope: None Aug 08 02:19:13.836817 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: User-Agent: python-novaclient Aug 08 02:19:13.836817 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-Auth-Token: gAAAAABi8HKhkiYVi4tNVDTGlsoXPbvMZl0VWb3FxWWicXszAXfgbblsx7edYMPsREhgt5h7dTE6WsDFpnrTHS3f9gfLfFPDm_ZM7cygnb3C5wFOQFHhPMZ8JpTx3RiugSfDN4GAKHlpqCgMrcoxeqW-CY0ZO0k6A0IZf8u54Ayl-d3rT9ZVRks Aug 08 02:19:13.836817 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-Domain-Id: None Aug 08 02:19:13.836817 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-Domain-Name: None Aug 08 02:19:13.836817 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-Identity-Status: Confirmed Aug 08 02:19:13.836817 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-Is-Admin-Project: True Aug 08 02:19:13.836817 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-Openstack-Nova-Api-Version: 2.90 Aug 08 02:19:13.836817 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-Project-Domain-Id: default Aug 08 02:19:13.836817 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-Project-Domain-Name: Default Aug 08 02:19:13.836817 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-Project-Id: 5f307f793c6c464eafa52c1ca17d9bc6 Aug 08 02:19:13.836817 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-Project-Name: admin Aug 08 02:19:13.836817 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-Role: reader,member,admin Aug 08 02:19:13.836817 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-Roles: reader,member,admin Aug 08 02:19:13.838418 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/5f307f793c6c464eafa52c1ca17d9bc6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/5f307f793c6c464eafa52c1ca17d9bc6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_5f307f793c6c464eafa52c1ca17d9bc6", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/5f307f793c6c464eafa52c1ca17d9bc6"}]}] Aug 08 02:19:13.838418 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-Tenant: admin Aug 08 02:19:13.838418 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-Tenant-Id: 5f307f793c6c464eafa52c1ca17d9bc6 Aug 08 02:19:13.838418 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-Tenant-Name: admin Aug 08 02:19:13.838418 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-User: admin Aug 08 02:19:13.838418 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-User-Domain-Id: default Aug 08 02:19:13.838418 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-User-Domain-Name: Default Aug 08 02:19:13.838418 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-User-Id: 979e7eb5d5594f0ea4d2839dfb577133 Aug 08 02:19:13.838418 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-User-Name: admin Aug 08 02:19:13.838418 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: method: > Aug 08 02:19:13.838418 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: exception: Flavor 84 could not be found. Aug 08 02:19:13.838418 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: action_args: {'id': '84'} Aug 08 02:19:13.838418 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: {{(pid=103872) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:19:13.838418 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:19:13.839302 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Aug 08 02:19:13.839302 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Aug 08 02:19:13.839302 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Aug 08 02:19:13.839302 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Aug 08 02:19:13.839302 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Aug 08 02:19:13.839302 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Aug 08 02:19:13.839302 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Aug 08 02:19:13.839302 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Aug 08 02:19:13.839302 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Aug 08 02:19:13.839302 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:19:13.839302 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Aug 08 02:19:13.839302 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Aug 08 02:19:13.839302 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Aug 08 02:19:13.839302 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi Aug 08 02:19:13.839302 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:19:13.839302 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi Aug 08 02:19:13.839302 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:19:13.839302 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:19:13.839302 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:19:13.840822 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:19:13.840822 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:19:13.840822 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:19:13.840822 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:19:13.840822 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Aug 08 02:19:13.840822 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:19:13.840822 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Aug 08 02:19:13.840822 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi Aug 08 02:19:13.840822 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:19:13.840822 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi Aug 08 02:19:13.840822 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:19:13.840822 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:19:13.840822 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:19:13.840822 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:19:13.840822 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:19:13.840822 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Aug 08 02:19:13.840822 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi Aug 08 02:19:13.840822 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi [None req-ef0e1299-b508-4af2-8745-b539ed1496af admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=103872) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:19:13.842077 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: INFO nova.api.openstack.requestlog [None req-ef0e1299-b508-4af2-8745-b539ed1496af admin admin] 10.222.0.27 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.90 time: 0.385403 Aug 08 02:19:13.842077 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: [pid: 103872|app: 0|req: 23/44] 10.222.0.27 () {60 vars in 1190 bytes} [Mon Aug 8 02:19:13 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 387 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Aug 08 02:19:13.848304 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi [None req-33f2f018-082a-45a9-96cc-2dd6d070ed0a admin admin] Calling method '>' {{(pid=103871) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:19:13.863580 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: INFO nova.api.openstack.requestlog [None req-33f2f018-082a-45a9-96cc-2dd6d070ed0a admin admin] 10.222.0.27 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.019305 Aug 08 02:19:13.864103 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: [pid: 103871|app: 0|req: 22/45] 10.222.0.27 () {60 vars in 1210 bytes} [Mon Aug 8 02:19:13 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:19:13.873334 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi [None req-6b355c68-0278-435d-b6aa-a8747817fc53 admin admin] Calling method '>' {{(pid=103872) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:19:13.890226 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: INFO nova.api.openstack.requestlog [None req-6b355c68-0278-435d-b6aa-a8747817fc53 admin admin] 10.222.0.27 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.020395 Aug 08 02:19:13.890750 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: [pid: 103872|app: 0|req: 24/46] 10.222.0.27 () {60 vars in 1210 bytes} [Mon Aug 8 02:19:13 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 21 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 08 02:19:13.900601 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi [None req-f9924303-44c8-4dfd-93c8-a269727b6910 admin admin] Calling method '>' {{(pid=103871) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:19:13.905945 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: INFO nova.api.openstack.wsgi [None req-f9924303-44c8-4dfd-93c8-a269727b6910 admin admin] HTTP exception thrown: Flavor 84 could not be found. Aug 08 02:19:13.907436 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi [None req-f9924303-44c8-4dfd-93c8-a269727b6910 admin admin] Request method failure captured: Aug 08 02:19:13.907436 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Aug 08 02:19:13.907436 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: Accept: application/json Aug 08 02:19:13.907436 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: Accept-Encoding: gzip, deflate Aug 08 02:19:13.907436 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: Connection: keep-alive Aug 08 02:19:13.907436 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: Content-Length: 0 Aug 08 02:19:13.907436 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: Host: 10.222.0.27 Aug 08 02:19:13.907436 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: Openstack-Api-Version: compute 2.90 Aug 08 02:19:13.907436 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: Openstack-System-Scope: None Aug 08 02:19:13.907436 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: User-Agent: python-novaclient Aug 08 02:19:13.907436 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-Auth-Token: gAAAAABi8HKhkiYVi4tNVDTGlsoXPbvMZl0VWb3FxWWicXszAXfgbblsx7edYMPsREhgt5h7dTE6WsDFpnrTHS3f9gfLfFPDm_ZM7cygnb3C5wFOQFHhPMZ8JpTx3RiugSfDN4GAKHlpqCgMrcoxeqW-CY0ZO0k6A0IZf8u54Ayl-d3rT9ZVRks Aug 08 02:19:13.907436 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-Domain-Id: None Aug 08 02:19:13.907436 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-Domain-Name: None Aug 08 02:19:13.907436 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-Identity-Status: Confirmed Aug 08 02:19:13.907436 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-Is-Admin-Project: True Aug 08 02:19:13.907436 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-Openstack-Nova-Api-Version: 2.90 Aug 08 02:19:13.907436 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-Project-Domain-Id: default Aug 08 02:19:13.907436 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-Project-Domain-Name: Default Aug 08 02:19:13.907436 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-Project-Id: 5f307f793c6c464eafa52c1ca17d9bc6 Aug 08 02:19:13.907436 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-Project-Name: admin Aug 08 02:19:13.907436 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-Role: reader,member,admin Aug 08 02:19:13.907436 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-Roles: reader,member,admin Aug 08 02:19:13.908960 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/5f307f793c6c464eafa52c1ca17d9bc6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/5f307f793c6c464eafa52c1ca17d9bc6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_5f307f793c6c464eafa52c1ca17d9bc6", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/5f307f793c6c464eafa52c1ca17d9bc6"}]}] Aug 08 02:19:13.908960 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-Tenant: admin Aug 08 02:19:13.908960 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-Tenant-Id: 5f307f793c6c464eafa52c1ca17d9bc6 Aug 08 02:19:13.908960 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-Tenant-Name: admin Aug 08 02:19:13.908960 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-User: admin Aug 08 02:19:13.908960 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-User-Domain-Id: default Aug 08 02:19:13.908960 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-User-Domain-Name: Default Aug 08 02:19:13.908960 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-User-Id: 979e7eb5d5594f0ea4d2839dfb577133 Aug 08 02:19:13.908960 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-User-Name: admin Aug 08 02:19:13.908960 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: method: > Aug 08 02:19:13.908960 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: exception: Flavor 84 could not be found. Aug 08 02:19:13.908960 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: action_args: {'id': '84'} Aug 08 02:19:13.908960 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: {{(pid=103871) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:19:13.908960 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:19:13.909814 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Aug 08 02:19:13.909814 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Aug 08 02:19:13.909814 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Aug 08 02:19:13.909814 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Aug 08 02:19:13.909814 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Aug 08 02:19:13.909814 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Aug 08 02:19:13.909814 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Aug 08 02:19:13.909814 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Aug 08 02:19:13.909814 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Aug 08 02:19:13.909814 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:19:13.909814 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Aug 08 02:19:13.909814 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Aug 08 02:19:13.909814 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Aug 08 02:19:13.909814 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi Aug 08 02:19:13.909814 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:19:13.909814 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi Aug 08 02:19:13.909814 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:19:13.909814 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:19:13.909814 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:19:13.910859 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:19:13.910859 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:19:13.910859 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:19:13.910859 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:19:13.910859 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Aug 08 02:19:13.910859 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:19:13.910859 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Aug 08 02:19:13.910859 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi Aug 08 02:19:13.910859 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:19:13.910859 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi Aug 08 02:19:13.910859 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:19:13.910859 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:19:13.910859 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:19:13.910859 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:19:13.910859 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:19:13.910859 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Aug 08 02:19:13.910859 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi Aug 08 02:19:13.910859 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi [None req-f9924303-44c8-4dfd-93c8-a269727b6910 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=103871) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:19:13.911875 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: INFO nova.api.openstack.requestlog [None req-f9924303-44c8-4dfd-93c8-a269727b6910 admin admin] 10.222.0.27 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.90 time: 0.011727 Aug 08 02:19:13.911875 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: [pid: 103871|app: 0|req: 23/47] 10.222.0.27 () {60 vars in 1190 bytes} [Mon Aug 8 02:19:13 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Aug 08 02:19:14.994026 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi [None req-6b355c68-0278-435d-b6aa-a8747817fc53 admin admin] Calling method '>' {{(pid=103872) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:19:14.995028 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: INFO nova.api.openstack.requestlog [None req-6b355c68-0278-435d-b6aa-a8747817fc53 admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001427 Aug 08 02:19:14.995419 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: [pid: 103872|app: 0|req: 25/48] 10.222.0.27 () {56 vars in 1074 bytes} [Mon Aug 8 02:19:14 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:19:15.525047 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi [None req-cbad3a05-0543-4170-bdf1-58fb61072b0e admin admin] Calling method '>' {{(pid=103871) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:19:15.530753 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: INFO nova.api.openstack.wsgi [None req-cbad3a05-0543-4170-bdf1-58fb61072b0e admin admin] HTTP exception thrown: Flavor 451 could not be found. Aug 08 02:19:15.532255 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi [None req-cbad3a05-0543-4170-bdf1-58fb61072b0e admin admin] Request method failure captured: Aug 08 02:19:15.532255 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Aug 08 02:19:15.532255 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: Accept: application/json Aug 08 02:19:15.532255 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: Accept-Encoding: gzip, deflate Aug 08 02:19:15.532255 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: Connection: keep-alive Aug 08 02:19:15.532255 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: Content-Length: 0 Aug 08 02:19:15.532255 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: Host: 10.222.0.27 Aug 08 02:19:15.532255 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: Openstack-Api-Version: compute 2.90 Aug 08 02:19:15.532255 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: Openstack-System-Scope: None Aug 08 02:19:15.532255 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: User-Agent: python-novaclient Aug 08 02:19:15.532255 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-Auth-Token: gAAAAABi8HKiUtbE_3oMQc8471EBSpec-mG3poGI-bhLd28kk0DZ4CRNo_NYzBGo4tGOUDWPS71Za6Cn4RPz3NdRG9WCqs_eLJxrvOUrb41nf4y1cKM4NWlqXJY7Vi0ZxNcKZ04Hvu2Pmr5rgJjedJIghUEgLZGfo6hZgzXbNauz1fyxhAYpdtc Aug 08 02:19:15.532255 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-Domain-Id: None Aug 08 02:19:15.532255 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-Domain-Name: None Aug 08 02:19:15.532255 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-Identity-Status: Confirmed Aug 08 02:19:15.532255 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-Is-Admin-Project: True Aug 08 02:19:15.532255 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-Openstack-Nova-Api-Version: 2.90 Aug 08 02:19:15.532255 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-Project-Domain-Id: default Aug 08 02:19:15.532255 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-Project-Domain-Name: Default Aug 08 02:19:15.532255 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-Project-Id: 5f307f793c6c464eafa52c1ca17d9bc6 Aug 08 02:19:15.532255 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-Project-Name: admin Aug 08 02:19:15.532255 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-Role: reader,member,admin Aug 08 02:19:15.532255 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-Roles: reader,member,admin Aug 08 02:19:15.533735 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/5f307f793c6c464eafa52c1ca17d9bc6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/5f307f793c6c464eafa52c1ca17d9bc6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_5f307f793c6c464eafa52c1ca17d9bc6", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/5f307f793c6c464eafa52c1ca17d9bc6"}]}] Aug 08 02:19:15.533735 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-Tenant: admin Aug 08 02:19:15.533735 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-Tenant-Id: 5f307f793c6c464eafa52c1ca17d9bc6 Aug 08 02:19:15.533735 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-Tenant-Name: admin Aug 08 02:19:15.533735 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-User: admin Aug 08 02:19:15.533735 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-User-Domain-Id: default Aug 08 02:19:15.533735 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-User-Domain-Name: Default Aug 08 02:19:15.533735 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-User-Id: 979e7eb5d5594f0ea4d2839dfb577133 Aug 08 02:19:15.533735 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: X-User-Name: admin Aug 08 02:19:15.533735 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: method: > Aug 08 02:19:15.533735 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: exception: Flavor 451 could not be found. Aug 08 02:19:15.533735 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: action_args: {'id': '451'} Aug 08 02:19:15.533735 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: {{(pid=103871) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:19:15.533735 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:19:15.534574 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Aug 08 02:19:15.534574 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Aug 08 02:19:15.534574 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Aug 08 02:19:15.534574 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Aug 08 02:19:15.534574 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Aug 08 02:19:15.534574 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Aug 08 02:19:15.534574 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Aug 08 02:19:15.534574 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Aug 08 02:19:15.534574 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Aug 08 02:19:15.534574 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:19:15.534574 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Aug 08 02:19:15.534574 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Aug 08 02:19:15.534574 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Aug 08 02:19:15.534574 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi Aug 08 02:19:15.534574 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:19:15.534574 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi Aug 08 02:19:15.534574 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:19:15.534574 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:19:15.534574 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:19:15.535810 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:19:15.535810 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:19:15.535810 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:19:15.535810 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:19:15.535810 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Aug 08 02:19:15.535810 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:19:15.535810 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Aug 08 02:19:15.535810 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi Aug 08 02:19:15.535810 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:19:15.535810 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi Aug 08 02:19:15.535810 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:19:15.535810 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:19:15.535810 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:19:15.535810 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:19:15.535810 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:19:15.535810 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Aug 08 02:19:15.535810 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: ERROR nova.api.openstack.wsgi Aug 08 02:19:15.535810 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi [None req-cbad3a05-0543-4170-bdf1-58fb61072b0e admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=103871) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:19:15.537135 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: INFO nova.api.openstack.requestlog [None req-cbad3a05-0543-4170-bdf1-58fb61072b0e admin admin] 10.222.0.27 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.90 time: 0.390624 Aug 08 02:19:15.537135 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: [pid: 103871|app: 0|req: 24/49] 10.222.0.27 () {60 vars in 1193 bytes} [Mon Aug 8 02:19:15 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 392 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Aug 08 02:19:15.543409 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi [None req-2d89ca94-9b8c-4c76-96af-2fe643963031 admin admin] Calling method '>' {{(pid=103872) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:19:15.558469 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: INFO nova.api.openstack.requestlog [None req-2d89ca94-9b8c-4c76-96af-2fe643963031 admin admin] 10.222.0.27 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.018347 Aug 08 02:19:15.558984 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: [pid: 103872|app: 0|req: 26/50] 10.222.0.27 () {60 vars in 1210 bytes} [Mon Aug 8 02:19:15 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:19:15.569098 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi [None req-d24ba093-dd2f-411f-8670-86ddac0fd714 admin admin] Calling method '>' {{(pid=103871) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:19:15.584984 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: INFO nova.api.openstack.requestlog [None req-d24ba093-dd2f-411f-8670-86ddac0fd714 admin admin] 10.222.0.27 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.020289 Aug 08 02:19:15.585518 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: [pid: 103871|app: 0|req: 25/51] 10.222.0.27 () {60 vars in 1210 bytes} [Mon Aug 8 02:19:15 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 21 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:19:15.594850 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi [None req-c09e361b-2e77-4dd0-9acf-ebd7e661dd22 admin admin] Calling method '>' {{(pid=103872) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:19:15.600305 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: INFO nova.api.openstack.wsgi [None req-c09e361b-2e77-4dd0-9acf-ebd7e661dd22 admin admin] HTTP exception thrown: Flavor 451 could not be found. Aug 08 02:19:15.601845 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi [None req-c09e361b-2e77-4dd0-9acf-ebd7e661dd22 admin admin] Request method failure captured: Aug 08 02:19:15.601845 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Aug 08 02:19:15.601845 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: Accept: application/json Aug 08 02:19:15.601845 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: Accept-Encoding: gzip, deflate Aug 08 02:19:15.601845 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: Connection: keep-alive Aug 08 02:19:15.601845 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: Content-Length: 0 Aug 08 02:19:15.601845 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: Host: 10.222.0.27 Aug 08 02:19:15.601845 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: Openstack-Api-Version: compute 2.90 Aug 08 02:19:15.601845 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: Openstack-System-Scope: None Aug 08 02:19:15.601845 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: User-Agent: python-novaclient Aug 08 02:19:15.601845 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-Auth-Token: gAAAAABi8HKiUtbE_3oMQc8471EBSpec-mG3poGI-bhLd28kk0DZ4CRNo_NYzBGo4tGOUDWPS71Za6Cn4RPz3NdRG9WCqs_eLJxrvOUrb41nf4y1cKM4NWlqXJY7Vi0ZxNcKZ04Hvu2Pmr5rgJjedJIghUEgLZGfo6hZgzXbNauz1fyxhAYpdtc Aug 08 02:19:15.601845 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-Domain-Id: None Aug 08 02:19:15.601845 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-Domain-Name: None Aug 08 02:19:15.601845 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-Identity-Status: Confirmed Aug 08 02:19:15.601845 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-Is-Admin-Project: True Aug 08 02:19:15.601845 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-Openstack-Nova-Api-Version: 2.90 Aug 08 02:19:15.601845 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-Project-Domain-Id: default Aug 08 02:19:15.601845 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-Project-Domain-Name: Default Aug 08 02:19:15.601845 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-Project-Id: 5f307f793c6c464eafa52c1ca17d9bc6 Aug 08 02:19:15.601845 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-Project-Name: admin Aug 08 02:19:15.601845 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-Role: reader,member,admin Aug 08 02:19:15.601845 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-Roles: reader,member,admin Aug 08 02:19:15.603411 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/5f307f793c6c464eafa52c1ca17d9bc6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/5f307f793c6c464eafa52c1ca17d9bc6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_5f307f793c6c464eafa52c1ca17d9bc6", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/5f307f793c6c464eafa52c1ca17d9bc6"}]}] Aug 08 02:19:15.603411 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-Tenant: admin Aug 08 02:19:15.603411 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-Tenant-Id: 5f307f793c6c464eafa52c1ca17d9bc6 Aug 08 02:19:15.603411 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-Tenant-Name: admin Aug 08 02:19:15.603411 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-User: admin Aug 08 02:19:15.603411 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-User-Domain-Id: default Aug 08 02:19:15.603411 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-User-Domain-Name: Default Aug 08 02:19:15.603411 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-User-Id: 979e7eb5d5594f0ea4d2839dfb577133 Aug 08 02:19:15.603411 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: X-User-Name: admin Aug 08 02:19:15.603411 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: method: > Aug 08 02:19:15.603411 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: exception: Flavor 451 could not be found. Aug 08 02:19:15.603411 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: action_args: {'id': '451'} Aug 08 02:19:15.603411 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: {{(pid=103872) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:19:15.603411 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:19:15.605224 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Aug 08 02:19:15.605224 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Aug 08 02:19:15.605224 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Aug 08 02:19:15.605224 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Aug 08 02:19:15.605224 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Aug 08 02:19:15.605224 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Aug 08 02:19:15.605224 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Aug 08 02:19:15.605224 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Aug 08 02:19:15.605224 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Aug 08 02:19:15.605224 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:19:15.605224 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Aug 08 02:19:15.605224 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Aug 08 02:19:15.605224 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Aug 08 02:19:15.605224 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi Aug 08 02:19:15.605224 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:19:15.605224 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi Aug 08 02:19:15.605224 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:19:15.605224 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:19:15.605224 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:19:15.606291 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:19:15.606291 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:19:15.606291 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:19:15.606291 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:19:15.606291 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Aug 08 02:19:15.606291 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:19:15.606291 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Aug 08 02:19:15.606291 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi Aug 08 02:19:15.606291 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:19:15.606291 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi Aug 08 02:19:15.606291 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:19:15.606291 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:19:15.606291 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:19:15.606291 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:19:15.606291 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:19:15.606291 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Aug 08 02:19:15.606291 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: ERROR nova.api.openstack.wsgi Aug 08 02:19:15.606291 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi [None req-c09e361b-2e77-4dd0-9acf-ebd7e661dd22 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=103872) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:19:15.607286 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: INFO nova.api.openstack.requestlog [None req-c09e361b-2e77-4dd0-9acf-ebd7e661dd22 admin admin] 10.222.0.27 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.90 time: 0.011941 Aug 08 02:19:15.607286 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: [pid: 103872|app: 0|req: 27/52] 10.222.0.27 () {60 vars in 1193 bytes} [Mon Aug 8 02:19:15 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Aug 08 02:19:16.603932 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi [None req-d24ba093-dd2f-411f-8670-86ddac0fd714 admin admin] Calling method '>' {{(pid=103871) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:19:16.604981 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: INFO nova.api.openstack.requestlog [None req-d24ba093-dd2f-411f-8670-86ddac0fd714 admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001432 Aug 08 02:19:16.605411 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: [pid: 103871|app: 0|req: 26/53] 10.222.0.27 () {56 vars in 1074 bytes} [Mon Aug 8 02:19:16 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:19:17.130004 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi [None req-a606368a-3ff7-4719-8344-a151c6413a72 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.nano", "ram": 256, "vcpus": 1, "disk": 1, "id": "42", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103872) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:19:17.223898 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: INFO nova.api.openstack.requestlog [None req-a606368a-3ff7-4719-8344-a151c6413a72 admin admin] 10.222.0.27 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.90 time: 0.475773 Aug 08 02:19:17.224434 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: [pid: 103872|app: 0|req: 28/54] 10.222.0.27 () {64 vars in 1235 bytes} [Mon Aug 8 02:19:16 2022] POST /compute/v2.1/flavors => generated 404 bytes in 477 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 08 02:19:18.202262 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi [None req-d24ba093-dd2f-411f-8670-86ddac0fd714 admin admin] Calling method '>' {{(pid=103871) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:19:18.203270 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: INFO nova.api.openstack.requestlog [None req-d24ba093-dd2f-411f-8670-86ddac0fd714 admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001432 Aug 08 02:19:18.203661 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: [pid: 103871|app: 0|req: 27/55] 10.222.0.27 () {56 vars in 1074 bytes} [Mon Aug 8 02:19:18 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:19:18.727427 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi [None req-ec1f0420-7d79-4c38-b294-76a965815dda admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.micro", "ram": 256, "vcpus": 1, "disk": 2, "id": "84", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103872) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:19:18.809145 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: INFO nova.api.openstack.requestlog [None req-ec1f0420-7d79-4c38-b294-76a965815dda admin admin] 10.222.0.27 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.90 time: 0.465422 Aug 08 02:19:18.809697 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: [pid: 103872|app: 0|req: 29/56] 10.222.0.27 () {64 vars in 1235 bytes} [Mon Aug 8 02:19:18 2022] POST /compute/v2.1/flavors => generated 405 bytes in 467 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 08 02:19:19.791846 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi [None req-d24ba093-dd2f-411f-8670-86ddac0fd714 admin admin] Calling method '>' {{(pid=103871) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:19:19.792919 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: INFO nova.api.openstack.requestlog [None req-d24ba093-dd2f-411f-8670-86ddac0fd714 admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001433 Aug 08 02:19:19.793335 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: [pid: 103871|app: 0|req: 28/57] 10.222.0.27 () {56 vars in 1074 bytes} [Mon Aug 8 02:19:19 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:19:20.310330 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi [None req-6ed42635-c753-48b7-910d-35f78e8ee3ed admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat", "ram": 512, "vcpus": 1, "disk": 1, "id": "451", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103872) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:19:20.391329 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: INFO nova.api.openstack.requestlog [None req-6ed42635-c753-48b7-910d-35f78e8ee3ed admin admin] 10.222.0.27 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.90 time: 0.458184 Aug 08 02:19:20.391988 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: [pid: 103872|app: 0|req: 30/58] 10.222.0.27 () {64 vars in 1235 bytes} [Mon Aug 8 02:19:19 2022] POST /compute/v2.1/flavors => generated 407 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 08 02:19:21.359102 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi [None req-d24ba093-dd2f-411f-8670-86ddac0fd714 admin admin] Calling method '>' {{(pid=103871) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:19:21.360129 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: INFO nova.api.openstack.requestlog [None req-d24ba093-dd2f-411f-8670-86ddac0fd714 admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001421 Aug 08 02:19:21.360547 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: [pid: 103871|app: 0|req: 29/59] 10.222.0.27 () {56 vars in 1074 bytes} [Mon Aug 8 02:19:21 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:19:21.883863 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi [None req-824b6e83-1c77-4e75-91ad-c0d7fa9512b2 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat2", "ram": 1024, "vcpus": 1, "disk": 1, "id": "452", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103872) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:19:21.967950 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: INFO nova.api.openstack.requestlog [None req-824b6e83-1c77-4e75-91ad-c0d7fa9512b2 admin admin] 10.222.0.27 "POST /compute/v2.1/flavors" status: 200 len: 409 microversion: 2.90 time: 0.468633 Aug 08 02:19:21.968514 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: [pid: 103872|app: 0|req: 31/60] 10.222.0.27 () {64 vars in 1235 bytes} [Mon Aug 8 02:19:21 2022] POST /compute/v2.1/flavors => generated 409 bytes in 470 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 08 02:19:22.925908 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi [None req-d24ba093-dd2f-411f-8670-86ddac0fd714 admin admin] Calling method '>' {{(pid=103871) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:19:22.926910 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: INFO nova.api.openstack.requestlog [None req-d24ba093-dd2f-411f-8670-86ddac0fd714 admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002553 Aug 08 02:19:22.927301 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: [pid: 103871|app: 0|req: 30/61] 10.222.0.27 () {56 vars in 1074 bytes} [Mon Aug 8 02:19:22 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:19:23.447160 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi [None req-bcf3dac5-ad64-41cb-af7b-3d87a40746de admin admin] Calling method '>' {{(pid=103872) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:19:23.467814 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: INFO nova.api.openstack.requestlog [None req-bcf3dac5-ad64-41cb-af7b-3d87a40746de admin admin] 10.222.0.27 "GET /compute/v2.1/flavors/detail" status: 200 len: 5777 microversion: 2.90 time: 0.401357 Aug 08 02:19:23.468352 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: [pid: 103872|app: 0|req: 32/62] 10.222.0.27 () {60 vars in 1202 bytes} [Mon Aug 8 02:19:23 2022] GET /compute/v2.1/flavors/detail => generated 5777 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:20:14.856655 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi [None req-d24ba093-dd2f-411f-8670-86ddac0fd714 admin admin] Calling method '>' {{(pid=103871) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:20:14.857653 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: INFO nova.api.openstack.requestlog [None req-d24ba093-dd2f-411f-8670-86ddac0fd714 admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001536 Aug 08 02:20:14.858059 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: [pid: 103871|app: 0|req: 31/63] 10.222.0.27 () {54 vars in 931 bytes} [Mon Aug 8 02:20:14 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:20:14.869044 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG dbcounter [-] [103872] Writing DB stats nova_api:SELECT=26,nova_api:INSERT=4 {{(pid=103872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:20:15.244945 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi [None req-cf453684-f806-49d2-8c72-fc97761ce924 admin admin] Calling method '>' {{(pid=103872) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:20:15.265419 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: INFO nova.api.openstack.requestlog [None req-cf453684-f806-49d2-8c72-fc97761ce924 admin admin] 10.222.0.27 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 5791 microversion: 2.61 time: 0.400775 Aug 08 02:20:15.265952 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: [pid: 103872|app: 0|req: 33/64] 10.222.0.27 () {60 vars in 1292 bytes} [Mon Aug 8 02:20:14 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 5791 bytes in 402 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:20:15.289968 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi [None req-58d4490f-2cbf-47da-82a3-1ecdbfe6e9d2 admin admin] Calling method '>' {{(pid=103871) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:20:15.293373 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG dbcounter [-] [103871] Writing DB stats nova_api:INSERT=2,nova_api:SELECT=22,nova_api:SAVEPOINT=1,nova_api:RELEASE=1 {{(pid=103871) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:20:15.299433 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: INFO nova.api.openstack.requestlog [None req-58d4490f-2cbf-47da-82a3-1ecdbfe6e9d2 admin admin] 10.222.0.27 "GET /compute/v2.1/flavors/42/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.013186 Aug 08 02:20:15.299941 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: [pid: 103871|app: 0|req: 32/65] 10.222.0.27 () {60 vars in 1283 bytes} [Mon Aug 8 02:20:15 2022] GET /compute/v2.1/flavors/42/os-extra_specs => generated 19 bytes in 14 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:20:15.311655 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi [None req-895b7ae1-ed49-4607-b344-75904f2efda8 admin admin] Calling method '>' {{(pid=103872) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:20:15.319733 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: INFO nova.api.openstack.requestlog [None req-895b7ae1-ed49-4607-b344-75904f2efda8 admin admin] 10.222.0.27 "GET /compute/v2.1/flavors/451/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011876 Aug 08 02:20:15.320260 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: [pid: 103872|app: 0|req: 34/66] 10.222.0.27 () {60 vars in 1286 bytes} [Mon Aug 8 02:20:15 2022] GET /compute/v2.1/flavors/451/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:20:15.332240 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi [None req-dd22ca3f-29af-4515-8279-20e0417084d3 admin admin] Calling method '>' {{(pid=103871) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:20:15.340082 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: INFO nova.api.openstack.requestlog [None req-dd22ca3f-29af-4515-8279-20e0417084d3 admin admin] 10.222.0.27 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011845 Aug 08 02:20:15.340587 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: [pid: 103871|app: 0|req: 33/67] 10.222.0.27 () {60 vars in 1286 bytes} [Mon Aug 8 02:20:15 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:20:15.352801 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi [None req-26459fe5-8969-4dd7-9aba-31aebd82cff5 admin admin] Calling method '>' {{(pid=103872) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:20:15.361542 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: INFO nova.api.openstack.requestlog [None req-26459fe5-8969-4dd7-9aba-31aebd82cff5 admin admin] 10.222.0.27 "GET /compute/v2.1/flavors/84/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011019 Aug 08 02:20:15.362061 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: [pid: 103872|app: 0|req: 35/68] 10.222.0.27 () {60 vars in 1283 bytes} [Mon Aug 8 02:20:15 2022] GET /compute/v2.1/flavors/84/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:23:17.777569 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi [None req-dd22ca3f-29af-4515-8279-20e0417084d3 admin admin] Calling method '>' {{(pid=103871) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:23:17.778583 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: INFO nova.api.openstack.requestlog [None req-dd22ca3f-29af-4515-8279-20e0417084d3 admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001813 Aug 08 02:23:17.778975 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: [pid: 103871|app: 0|req: 34/69] 10.222.0.27 () {56 vars in 1074 bytes} [Mon Aug 8 02:23:17 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 08 02:23:17.923320 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG dbcounter [-] [103872] Writing DB stats nova_api:SELECT=6 {{(pid=103872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:23:18.080076 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi [None req-9e8c55f2-534e-491d-ae30-1364f33d1ad3 admin admin] Calling method '>' {{(pid=103872) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:23:18.082426 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:23:18.082426 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: warnings.warn(msg) Aug 08 02:23:18.083546 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG oslo_concurrency.lockutils [None req-9e8c55f2-534e-491d-ae30-1364f33d1ad3 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103872) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:23:18.084023 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG oslo_concurrency.lockutils [None req-9e8c55f2-534e-491d-ae30-1364f33d1ad3 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=103872) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:23:18.084453 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG oslo_concurrency.lockutils [None req-9e8c55f2-534e-491d-ae30-1364f33d1ad3 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103872) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:23:18.085194 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG oslo_concurrency.lockutils [None req-9e8c55f2-534e-491d-ae30-1364f33d1ad3 admin admin] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103872) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:23:18.085640 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG oslo_concurrency.lockutils [None req-9e8c55f2-534e-491d-ae30-1364f33d1ad3 admin admin] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103872) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:23:18.086076 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG oslo_concurrency.lockutils [None req-9e8c55f2-534e-491d-ae30-1364f33d1ad3 admin admin] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103872) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:23:18.120175 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: INFO nova.api.openstack.requestlog [None req-9e8c55f2-534e-491d-ae30-1364f33d1ad3 admin admin] 10.222.0.27 "GET /compute/v2.1/os-services" status: 200 len: 794 microversion: 2.90 time: 0.198439 Aug 08 02:23:18.120753 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: [pid: 103872|app: 0|req: 36/70] 10.222.0.27 () {60 vars in 1193 bytes} [Mon Aug 8 02:23:17 2022] GET /compute/v2.1/os-services => generated 794 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 08 02:23:18.957940 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi [None req-dd22ca3f-29af-4515-8279-20e0417084d3 admin admin] Calling method '>' {{(pid=103871) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:23:18.958947 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: INFO nova.api.openstack.requestlog [None req-dd22ca3f-29af-4515-8279-20e0417084d3 admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001462 Aug 08 02:23:18.959337 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: [pid: 103871|app: 0|req: 35/71] 10.222.0.27 () {56 vars in 1074 bytes} [Mon Aug 8 02:23:18 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:23:19.151327 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi [None req-95732754-4ffe-4adf-9de2-14e5f97439d4 admin admin] Calling method '>' {{(pid=103872) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:23:19.153730 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:23:19.153730 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: warnings.warn(msg) Aug 08 02:23:19.155467 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG oslo_concurrency.lockutils [None req-95732754-4ffe-4adf-9de2-14e5f97439d4 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103872) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:23:19.155940 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG oslo_concurrency.lockutils [None req-95732754-4ffe-4adf-9de2-14e5f97439d4 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=103872) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:23:19.156369 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG oslo_concurrency.lockutils [None req-95732754-4ffe-4adf-9de2-14e5f97439d4 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103872) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:23:19.157098 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG oslo_concurrency.lockutils [None req-95732754-4ffe-4adf-9de2-14e5f97439d4 admin admin] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103872) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:23:19.157542 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG oslo_concurrency.lockutils [None req-95732754-4ffe-4adf-9de2-14e5f97439d4 admin admin] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103872) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:23:19.157955 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG oslo_concurrency.lockutils [None req-95732754-4ffe-4adf-9de2-14e5f97439d4 admin admin] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103872) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:23:19.178023 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: INFO nova.api.openstack.requestlog [None req-95732754-4ffe-4adf-9de2-14e5f97439d4 admin admin] 10.222.0.27 "GET /compute/v2.1/os-services" status: 200 len: 794 microversion: 2.90 time: 0.078615 Aug 08 02:23:19.178511 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: [pid: 103872|app: 0|req: 37/72] 10.222.0.27 () {60 vars in 1193 bytes} [Mon Aug 8 02:23:19 2022] GET /compute/v2.1/os-services => generated 794 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 08 02:23:30.431901 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi [None req-dd22ca3f-29af-4515-8279-20e0417084d3 admin admin] Calling method '>' {{(pid=103871) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:23:30.432948 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: INFO nova.api.openstack.requestlog [None req-dd22ca3f-29af-4515-8279-20e0417084d3 admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001416 Aug 08 02:23:30.433370 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: [pid: 103871|app: 0|req: 36/73] 10.222.0.27 () {56 vars in 1074 bytes} [Mon Aug 8 02:23:30 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:23:30.581226 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG dbcounter [-] [103872] Writing DB stats nova_cell0:SELECT=4 {{(pid=103872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:23:30.581960 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG dbcounter [-] [103872] Writing DB stats nova_cell1:SELECT=4 {{(pid=103872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:23:30.582601 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG dbcounter [-] [103872] Writing DB stats nova_api:SELECT=8 {{(pid=103872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:23:30.630742 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi [None req-8f21673d-0a0e-47fc-88c8-89775435f40a admin admin] Calling method '>' {{(pid=103872) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:23:30.633177 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:23:30.633177 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: warnings.warn(msg) Aug 08 02:23:30.634281 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG oslo_concurrency.lockutils [None req-8f21673d-0a0e-47fc-88c8-89775435f40a admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103872) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:23:30.634800 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG oslo_concurrency.lockutils [None req-8f21673d-0a0e-47fc-88c8-89775435f40a admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=103872) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:23:30.635234 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG oslo_concurrency.lockutils [None req-8f21673d-0a0e-47fc-88c8-89775435f40a admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103872) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:23:30.635954 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG oslo_concurrency.lockutils [None req-8f21673d-0a0e-47fc-88c8-89775435f40a admin admin] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103872) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:23:30.636407 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG oslo_concurrency.lockutils [None req-8f21673d-0a0e-47fc-88c8-89775435f40a admin admin] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103872) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:23:30.636866 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG oslo_concurrency.lockutils [None req-8f21673d-0a0e-47fc-88c8-89775435f40a admin admin] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103872) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:23:30.664126 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: INFO nova.api.openstack.requestlog [None req-8f21673d-0a0e-47fc-88c8-89775435f40a admin admin] 10.222.0.27 "GET /compute/v2.1/os-services" status: 200 len: 1038 microversion: 2.90 time: 0.087235 Aug 08 02:23:30.664615 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: [pid: 103872|app: 0|req: 38/74] 10.222.0.27 () {60 vars in 1193 bytes} [Mon Aug 8 02:23:30 2022] GET /compute/v2.1/os-services => generated 1038 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:23:31.488723 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: DEBUG nova.api.openstack.wsgi [None req-dd22ca3f-29af-4515-8279-20e0417084d3 admin admin] Calling method '>' {{(pid=103871) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:23:31.489734 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: INFO nova.api.openstack.requestlog [None req-dd22ca3f-29af-4515-8279-20e0417084d3 admin admin] 10.222.0.27 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001449 Aug 08 02:23:31.490152 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: [pid: 103871|app: 0|req: 37/75] 10.222.0.27 () {56 vars in 1074 bytes} [Mon Aug 8 02:23:31 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:23:31.680507 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG nova.api.openstack.wsgi [None req-ecd108b5-81c6-44cd-beec-ba51d83500e4 admin admin] Calling method '>' {{(pid=103872) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:23:31.682927 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:23:31.682927 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: warnings.warn(msg) Aug 08 02:23:31.684288 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG oslo_concurrency.lockutils [None req-ecd108b5-81c6-44cd-beec-ba51d83500e4 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103872) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:23:31.684801 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG oslo_concurrency.lockutils [None req-ecd108b5-81c6-44cd-beec-ba51d83500e4 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103872) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:23:31.685798 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG oslo_concurrency.lockutils [None req-ecd108b5-81c6-44cd-beec-ba51d83500e4 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103872) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:23:31.686507 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG oslo_concurrency.lockutils [None req-ecd108b5-81c6-44cd-beec-ba51d83500e4 admin admin] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103872) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:23:31.686954 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG oslo_concurrency.lockutils [None req-ecd108b5-81c6-44cd-beec-ba51d83500e4 admin admin] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103872) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:23:31.687373 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: DEBUG oslo_concurrency.lockutils [None req-ecd108b5-81c6-44cd-beec-ba51d83500e4 admin admin] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103872) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:23:31.709998 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: INFO nova.api.openstack.requestlog [None req-ecd108b5-81c6-44cd-beec-ba51d83500e4 admin admin] 10.222.0.27 "GET /compute/v2.1/os-services" status: 200 len: 1038 microversion: 2.90 time: 0.080185 Aug 08 02:23:31.710480 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: [pid: 103872|app: 0|req: 39/76] 10.222.0.27 () {60 vars in 1193 bytes} [Mon Aug 8 02:23:31 2022] GET /compute/v2.1/os-services => generated 1038 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 08 02:24:28.542615 dv-cinder-740146-18-iscsi systemd[1]: Stopping Devstack devstack@n-api.service... Aug 08 02:24:28.543153 dv-cinder-740146-18-iscsi devstack@n-api.service[103870]: Mon Aug 8 02:24:28 2022 - graceful shutdown triggered... Aug 08 02:24:28.543616 dv-cinder-740146-18-iscsi devstack@n-api.service[103871]: Gracefully killing worker 1 (pid: 103871)... Aug 08 02:24:28.544040 dv-cinder-740146-18-iscsi devstack@n-api.service[103872]: Gracefully killing worker 2 (pid: 103872)... Aug 08 02:24:29.544077 dv-cinder-740146-18-iscsi devstack@n-api.service[103870]: worker 1 buried after 1 seconds Aug 08 02:24:29.544595 dv-cinder-740146-18-iscsi devstack@n-api.service[103870]: worker 2 buried after 1 seconds Aug 08 02:24:29.544595 dv-cinder-740146-18-iscsi devstack@n-api.service[103870]: goodbye to uWSGI. Aug 08 02:24:29.546061 dv-cinder-740146-18-iscsi systemd[1]: devstack@n-api.service: Succeeded. Aug 08 02:24:29.546635 dv-cinder-740146-18-iscsi systemd[1]: Stopped Devstack devstack@n-api.service. Aug 08 02:24:29.549529 dv-cinder-740146-18-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Aug 08 02:24:29.558981 dv-cinder-740146-18-iscsi devstack@n-api.service[118332]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Aug 08 02:24:29.560920 dv-cinder-740146-18-iscsi devstack@n-api.service[118332]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Aug 8 02:24:29 2022] *** Aug 08 02:24:29.560920 dv-cinder-740146-18-iscsi devstack@n-api.service[118332]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 08 02:24:29.560920 dv-cinder-740146-18-iscsi devstack@n-api.service[118332]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 08 02:24:29.560920 dv-cinder-740146-18-iscsi devstack@n-api.service[118332]: nodename: dv-cinder-740146-18-iscsi Aug 08 02:24:29.560920 dv-cinder-740146-18-iscsi devstack@n-api.service[118332]: machine: x86_64 Aug 08 02:24:29.560920 dv-cinder-740146-18-iscsi devstack@n-api.service[118332]: clock source: unix Aug 08 02:24:29.560920 dv-cinder-740146-18-iscsi devstack@n-api.service[118332]: pcre jit disabled Aug 08 02:24:29.560920 dv-cinder-740146-18-iscsi devstack@n-api.service[118332]: detected number of CPU cores: 4 Aug 08 02:24:29.560920 dv-cinder-740146-18-iscsi devstack@n-api.service[118332]: current working directory: / Aug 08 02:24:29.560920 dv-cinder-740146-18-iscsi devstack@n-api.service[118332]: detected binary path: /usr/bin/uwsgi-core Aug 08 02:24:29.562068 dv-cinder-740146-18-iscsi devstack@n-api.service[118332]: your processes number limit is 47680 Aug 08 02:24:29.562068 dv-cinder-740146-18-iscsi devstack@n-api.service[118332]: your memory page size is 4096 bytes Aug 08 02:24:29.562068 dv-cinder-740146-18-iscsi devstack@n-api.service[118332]: detected max file descriptor number: 2048 Aug 08 02:24:29.562068 dv-cinder-740146-18-iscsi devstack@n-api.service[118332]: lock engine: pthread robust mutexes Aug 08 02:24:29.562068 dv-cinder-740146-18-iscsi devstack@n-api.service[118332]: thunder lock: enabled Aug 08 02:24:29.562068 dv-cinder-740146-18-iscsi devstack@n-api.service[118332]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Aug 08 02:24:29.562068 dv-cinder-740146-18-iscsi devstack@n-api.service[118332]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 08 02:24:29.601975 dv-cinder-740146-18-iscsi devstack@n-api.service[118332]: Python main interpreter initialized at 0x55d63d2e19c0 Aug 08 02:24:29.601975 dv-cinder-740146-18-iscsi devstack@n-api.service[118332]: python threads support enabled Aug 08 02:24:29.601975 dv-cinder-740146-18-iscsi devstack@n-api.service[118332]: your server socket listen backlog is limited to 100 connections Aug 08 02:24:29.601975 dv-cinder-740146-18-iscsi devstack@n-api.service[118332]: your mercy for graceful operations on workers is 90 seconds Aug 08 02:24:29.601975 dv-cinder-740146-18-iscsi devstack@n-api.service[118332]: mapped 403077 bytes (393 KB) for 2 cores Aug 08 02:24:29.601975 dv-cinder-740146-18-iscsi devstack@n-api.service[118332]: *** Operational MODE: preforking *** Aug 08 02:24:29.605643 dv-cinder-740146-18-iscsi devstack@n-api.service[118332]: *** uWSGI is running in multiple interpreter mode *** Aug 08 02:24:29.605643 dv-cinder-740146-18-iscsi devstack@n-api.service[118332]: spawned uWSGI master process (pid: 118332) Aug 08 02:24:29.605643 dv-cinder-740146-18-iscsi devstack@n-api.service[118332]: spawned uWSGI worker 1 (pid: 118351, cores: 1) Aug 08 02:24:29.605643 dv-cinder-740146-18-iscsi devstack@n-api.service[118332]: spawned uWSGI worker 2 (pid: 118352, cores: 1) Aug 08 02:24:29.605643 dv-cinder-740146-18-iscsi devstack@n-api.service[118332]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 08 02:24:29.602367 dv-cinder-740146-18-iscsi systemd[1]: Started Devstack devstack@n-api.service. Aug 08 02:24:30.282713 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Aug 08 02:24:30.294889 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Aug 08 02:24:31.479212 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 08 02:24:31.479212 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: from cryptography.utils import int_from_bytes Aug 08 02:24:31.479831 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 08 02:24:31.479831 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: from cryptography.utils import int_from_bytes Aug 08 02:24:31.481315 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 08 02:24:31.481315 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: from cryptography.utils import int_from_bytes Aug 08 02:24:31.481775 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 08 02:24:31.481775 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: from cryptography.utils import int_from_bytes Aug 08 02:24:33.844304 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 08 02:24:33.844877 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 08 02:24:33.845131 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 08 02:24:33.845499 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 08 02:24:33.845758 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 08 02:24:33.846152 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.846490 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.846808 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.847133 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.847474 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.847790 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.848115 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.848438 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.848875 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.849142 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.849463 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.849793 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-740146-18-iscsi {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.850119 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.850430 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.850744 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.851064 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.851375 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.851703 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.852012 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.852511 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.853934 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.853934 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.853934 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.853934 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.854305 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.854497 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.855000 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.855325 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.855645 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.855951 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.856358 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-740146-18-iscsi {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.856713 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.857013 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.857338 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.857757 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.858731 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.859049 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.859377 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.859705 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.860019 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.860362 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.860705 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.861032 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.861356 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.861664 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.861975 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.862289 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.862611 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.862925 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.863236 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.863487 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.863797 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.864124 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.864446 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.864793 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.865052 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.865306 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.865627 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.865952 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.866263 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.866573 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.866881 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.867192 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.867509 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.867817 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.868124 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.868432 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.868539 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 08 02:24:33.868969 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 08 02:24:33.869050 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.869129 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.869284 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 08 02:24:33.869379 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.869646 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 08 02:24:33.869747 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.869912 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 08 02:24:33.870039 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.870303 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.870399 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.27 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.870621 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.27 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.870725 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.870931 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.871033 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.871255 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.871347 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.871575 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.871667 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.871887 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.871994 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.872208 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.872317 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.872528 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.872617 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.872863 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.873023 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.873155 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.873261 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.873456 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.873576 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.873880 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.873971 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-740146-18-iscsi {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.874252 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.874360 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.874567 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.874667 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.874867 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.874974 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.875176 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.875274 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.875486 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.875590 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.875810 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.875906 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.876119 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.876220 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.876430 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.876623 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.876866 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.876962 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.877064 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.877262 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.877371 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.877580 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.877682 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.877928 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.878046 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.878251 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.878364 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.878562 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.878662 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.878871 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.878971 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.879184 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.879296 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.879493 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.879609 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.879802 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.879899 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.880126 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.880236 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-740146-18-iscsi {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.880445 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.880552 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.880862 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.881012 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.881152 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.881290 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.881447 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.881597 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.881738 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.881899 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.882052 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.882200 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.882339 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.882477 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.882626 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.882775 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.882927 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.883130 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.883287 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.883436 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.883586 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.883739 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.883886 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.884034 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.884201 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.884357 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.884504 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.884652 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.884827 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.884977 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.885264 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.885410 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.885559 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.885717 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.885857 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.885994 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.886142 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.886289 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.886443 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.886591 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.886738 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.886903 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.887052 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.887192 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.888251 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.888431 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.888581 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.888581 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.888920 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.888920 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.889275 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.889451 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.889606 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.889747 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.889888 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.890041 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.890210 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.890450 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.890730 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.890884 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.891033 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.891204 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.891353 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.891493 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.891633 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.891779 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.891779 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.892088 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.892243 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.892395 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.892546 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.892712 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.892938 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.893125 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.893278 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.893429 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.893578 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.893718 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.893871 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.894011 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.894168 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.894310 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.894310 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.894604 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.27 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.894604 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.27 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.894886 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.895026 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.895190 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.895475 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.895634 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.895783 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.895924 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.896063 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.896201 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.896340 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.896502 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.896651 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.896826 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.896973 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.897276 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.897432 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.897587 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.897740 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.897887 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.898046 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.898460 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.898605 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.898777 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.898927 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.899075 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.899215 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.899362 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.899511 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.899669 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.899823 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.899962 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.900108 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.900256 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.900427 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.900637 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.900897 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.901044 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.901183 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.901332 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.901486 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.901625 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.901764 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.901924 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.902072 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.902219 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.902372 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.902511 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.902658 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.902658 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.902944 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.903096 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.903289 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.903430 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.903577 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.903727 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.903868 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.904021 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.904168 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.904308 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.904447 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.904586 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.905284 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.905595 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.905877 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.906032 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.906180 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.906321 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.906461 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.906611 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.906759 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.906900 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.907050 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.907191 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.907340 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.907486 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.907626 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.907626 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.907899 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.908048 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.908206 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.908579 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.908762 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.908924 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.909080 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.909236 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.909382 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.909541 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.909688 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.909828 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.909976 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.910125 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.910923 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.911155 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.911311 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.911452 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.911600 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.911750 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.911903 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.912052 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.912210 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.912353 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.912513 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.912714 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.912877 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.913018 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.913157 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.913346 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.913505 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.913505 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.913780 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.913918 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.914057 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.914207 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.914360 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.914508 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.914666 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.914806 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.914944 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.915096 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.915244 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.915384 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.915549 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.915828 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.915975 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.916131 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:24:33.916282 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.916422 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.916572 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.916729 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.916874 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.917026 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.917174 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.917314 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.917460 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.917460 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.917771 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.917926 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.918084 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.918233 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.918380 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.918893 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.919058 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.919206 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.919347 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.919347 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.919628 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.919774 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.919928 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.920075 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.920223 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.920417 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.921441 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.921592 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.921819 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.921971 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.922118 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.922260 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.922399 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.922399 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.922699 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.922841 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.922988 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.923142 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.923289 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.923487 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.923630 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.923776 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.923917 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.924079 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.924229 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.924382 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.924532 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.924690 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.924849 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.924990 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.924990 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.925274 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.925422 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.925569 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.925717 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.925899 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.926055 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.926206 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.926346 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.926495 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.926636 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.926775 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.926928 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.927097 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.927097 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.927097 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.927097 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.927097 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.929127 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.929218 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.929293 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.929371 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.929450 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.929529 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.929615 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.929696 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.929774 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.929854 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.929927 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.930007 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.930116 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.930191 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.930269 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.930340 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.930418 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.930502 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.930575 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.930654 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.930726 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.930796 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.930866 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.930945 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.931025 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.931097 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.931169 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.931239 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.931324 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.931423 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.931503 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.931583 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.931655 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.931724 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.931801 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.931886 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.931965 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.932036 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.932036 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.932202 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.932285 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.932357 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.932428 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.932504 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.932575 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.932651 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.932770 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.932845 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.932923 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.932995 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.933066 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.933142 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.933223 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.933295 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.933534 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.933881 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.934044 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.934141 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.934363 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.934484 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.934678 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.934915 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.935166 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.935335 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.936165 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.936307 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.936410 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.936498 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.936590 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.936723 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.936723 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.936907 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.937000 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.937093 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.937178 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.937276 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.937369 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.937460 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.937544 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.937734 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.937850 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.938054 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.938156 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.938376 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.938473 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.938715 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.938821 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.939037 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.939138 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.939356 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.939457 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.939667 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.939772 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.939977 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.940076 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.940294 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.940396 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.940613 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.940896 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.940988 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.941080 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.941256 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.941358 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.941570 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.941673 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.941888 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.941979 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.942200 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.942291 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.942508 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.942600 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.942814 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.942913 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.943127 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.943219 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.943439 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.943533 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.943749 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.943850 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.944059 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.944151 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.944370 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.944480 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.944700 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.944806 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.945009 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.945110 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.945326 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.945430 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.945634 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.945726 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.945941 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.946061 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.946249 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.946342 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.946557 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.946650 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.946891 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.946997 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.947203 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.947293 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.947516 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.947704 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.947856 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.948091 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.948245 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.948388 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.948536 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.948733 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.948883 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.949035 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.949192 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.949349 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.949525 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.949830 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.950020 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.950254 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.950500 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.950663 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.950817 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.950972 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.951146 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.951295 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.951448 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.951650 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.952034 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.952272 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.952441 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.952600 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.952807 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.952966 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.953124 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.953271 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.953271 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.953623 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.953795 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.953952 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.954100 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.954249 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.954405 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.954560 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.954723 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.955462 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.955631 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.955794 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.955969 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.956119 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.956397 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.956557 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.956732 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.956903 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.957051 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.957051 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.957051 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.957468 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.957615 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.957774 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.957922 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.958067 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.958234 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.958389 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.958545 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.958702 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.958857 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.959013 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.959181 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.959340 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.959489 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.959661 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.959901 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.960078 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.960257 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.960644 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.960871 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.961046 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.961211 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.961367 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.961531 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.961687 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.961835 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.962127 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.962282 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.962448 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.962448 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.962736 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.962894 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.963056 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.963227 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.963395 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.963550 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.963747 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.963920 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.964074 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.964245 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.964411 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.964567 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.964732 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.964909 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.965076 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.965233 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.965382 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.965536 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.965716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.965884 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.966064 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.966229 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.966396 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.966396 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.966763 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.966913 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.967077 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.967235 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.967970 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.968227 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.968415 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.968586 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.968778 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.968944 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.969102 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.969250 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.969404 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.969575 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.969737 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.969928 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.970163 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.970317 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.970464 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.970813 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.970982 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.971131 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.971295 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.971443 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.971596 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.971752 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.971909 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.972204 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.972386 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.972544 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.972722 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.972894 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.973053 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.973211 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.973366 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.973537 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.973687 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.973889 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.974043 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.974191 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.974337 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.974495 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.974650 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.974842 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.974998 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.975154 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.975308 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.975456 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.975602 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.975757 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.975905 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.976059 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.976208 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.976353 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.976508 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.976780 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.976936 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.977084 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.977239 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.977386 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.977544 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.977699 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.977854 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.978009 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.978156 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.978312 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.978485 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.978634 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.978781 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.978926 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.979072 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.979219 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.979376 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.979376 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.979662 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.979809 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.979955 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.980109 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.980358 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.980530 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.980709 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.980967 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.981123 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.981282 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.981282 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.981570 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.981729 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.981883 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.982038 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.982403 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.982564 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.982713 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.982912 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.983683 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.983973 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.984141 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.984290 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.984445 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.984594 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.984804 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.984955 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.985111 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.985258 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.985413 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.985586 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.985737 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.985885 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.986040 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.986194 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.986349 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.986506 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.986654 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.986887 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.987042 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.987190 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.987345 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.987506 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.987665 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.987813 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.987968 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.988123 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.988279 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.988436 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.988585 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.988765 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.988918 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.989065 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.989288 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.989437 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.989584 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.989749 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.989906 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.990064 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.990221 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.990384 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.990613 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.990784 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.990941 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.991652 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.991774 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.991857 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.991941 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.992017 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.992098 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.992178 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.992265 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.992339 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.992418 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.992491 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.992562 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.992646 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.992751 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.992825 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.992897 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.992968 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.993038 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.993128 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.993204 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.993204 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.993352 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.993432 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.993505 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.993589 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.993663 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.993741 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.993814 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.993892 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.993973 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.994054 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.994134 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.994207 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.994278 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.994357 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.994443 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.994522 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.994595 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.994666 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.994743 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.994824 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.994910 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.994997 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.996066 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.996165 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.996267 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.996346 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.996433 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.996515 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.996589 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.996660 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.996753 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.996826 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.996912 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.996912 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.997053 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.997124 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.997195 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.997287 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.997362 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.997433 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.997503 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.997574 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.997652 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.997734 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.997814 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.997887 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.997958 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.998028 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.998097 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.998177 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.998275 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.998352 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.998423 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.998495 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.998591 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.998770 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.998970 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.999130 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.999292 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.999453 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.999624 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.999781 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:33.999955 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.000104 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.000273 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.000434 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.000617 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.000818 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.000967 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.000967 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.001388 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.001388 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.001692 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.001852 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.001999 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.002145 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.002300 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.002449 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.002606 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.002753 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.002908 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.003063 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.003218 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.003380 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.003529 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.003683 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.003847 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.004007 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.004164 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.004321 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.004477 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.004634 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.004634 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.004966 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.005377 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.005697 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.006009 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.006436 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.006753 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.006927 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.007083 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.007383 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.007730 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.008038 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.008356 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.008702 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.009015 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.009307 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.009406 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.009657 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.009760 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.009975 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.010294 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.010604 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.010795 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.010934 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.011134 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.011309 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.011569 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.011725 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.011874 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.012021 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.012168 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.012315 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.012505 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.012695 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.012862 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.013021 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.013177 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.013339 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.013487 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.013651 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.013936 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.014092 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.014275 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.014433 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.014582 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.014737 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.014884 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.015032 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.015230 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.015395 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.015543 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.015698 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.015845 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.015991 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.016138 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.016299 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.016568 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.016750 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.016930 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.017083 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.017314 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.017464 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.017610 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.017765 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.017913 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.018067 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.018230 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.018386 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.018616 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.018777 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.018971 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.019133 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.019292 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.019448 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.019596 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.019742 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.019888 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.020034 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.020190 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.020346 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.020494 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.020640 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.020822 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.020995 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.021172 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.021323 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.021478 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.021732 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.021889 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.022045 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.022193 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.022349 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.022517 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.022679 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.022840 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.022989 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.023153 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.023302 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.023450 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.023606 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.023755 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.023927 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.024146 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.024319 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.024483 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.024646 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.024843 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.025000 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.025158 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.025158 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.025494 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.025656 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.025843 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.026004 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.026153 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.026299 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.026455 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.026693 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.026863 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.027016 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.027163 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.027319 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.027476 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.027625 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.027772 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.027926 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.028081 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.028237 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.028393 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.028393 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.028796 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.028959 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.029108 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.029262 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.029434 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.029594 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.030262 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.030420 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.030575 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.030725 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.030882 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.031030 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.031177 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.031324 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.031479 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.031677 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.031933 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.032210 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.032375 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.032531 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.032722 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.032893 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.033043 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.033198 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.033347 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.033509 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.033658 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.033818 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.033974 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.034158 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.034380 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.034541 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.034697 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.034859 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.035009 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.035156 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.035303 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.035503 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.035662 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.035818 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.035968 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.036115 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.036272 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.036419 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.036576 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.036832 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.036997 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.037155 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.037311 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.037469 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.037627 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.037775 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.037930 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.038080 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.038227 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.038395 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.038551 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.038707 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.038949 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.039105 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.039253 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.039410 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.039566 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.039714 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.039862 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.040017 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.040175 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.040417 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.040575 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.040823 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.040985 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.041134 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.041282 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.041439 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.041587 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.041742 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.041890 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.042139 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.042305 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.042463 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.042629 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.042786 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.042980 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.043142 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.043310 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.043467 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.043630 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.043778 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.043925 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.044082 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.044229 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.044394 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.044625 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.044812 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.044971 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.045134 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.045282 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.046013 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.046182 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.046331 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.046479 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.046647 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.046814 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.047039 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.047205 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.047374 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.047534 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.047696 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.047851 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.048007 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.048163 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.048311 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.048467 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.048623 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.048817 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.049058 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.049220 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.049386 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.049545 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.049700 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.049856 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.050003 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.050158 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.050158 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.050451 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.050600 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.050761 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.050909 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.051055 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.051211 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.051372 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.051530 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.051677 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.051826 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.051980 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.052155 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.052857 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.053021 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.053171 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.053318 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.053465 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.053626 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.053798 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.053955 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.054112 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.054112 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.054403 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.054560 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.054802 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.054958 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.055107 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.055262 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.055421 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.055622 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.055784 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.055945 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.056093 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.056249 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.056406 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.056406 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.056707 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.056873 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.057040 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.057266 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.057434 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.057584 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.057739 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.057895 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.057895 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.058203 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.058360 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.058360 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.058909 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.059086 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.059311 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.059475 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.059631 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.059780 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.059935 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.060084 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.060251 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.060415 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.060565 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.060746 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.060958 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.061156 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.061321 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.061480 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.061629 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.061629 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.061908 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.062054 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.062202 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.062368 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.062516 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.062663 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.062809 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.063076 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.063245 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.063401 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.063556 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.063714 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.063869 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.064025 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.064182 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.064182 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.064479 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.064635 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.064820 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.065038 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.065204 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.065355 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.065510 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.065698 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.065859 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.066016 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.066174 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.066174 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.066456 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.066622 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.066779 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.066937 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.067093 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.067266 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.067494 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.067647 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.067795 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.067951 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.068106 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.068254 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.068411 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.068577 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.068756 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.068920 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.069076 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.069232 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.069462 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.069620 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.069776 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.069933 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.070088 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.070236 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.070383 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.070530 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.070701 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.070858 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.071005 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.071152 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.071306 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.071461 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.071687 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.071836 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.071983 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.071983 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.072262 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.072262 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.072565 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.072776 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.072930 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.072930 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.073311 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.073482 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.073632 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.073791 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.073947 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.074103 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.074250 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.074411 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.074569 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.074716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.074863 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.075028 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.075253 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.075420 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.075569 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.075759 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.075912 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.076061 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.076227 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.076376 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.076522 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.076725 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.076976 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.077237 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.077408 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.077656 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.077812 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.077812 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.078102 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.078250 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.078408 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.078555 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.078703 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.078861 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.079052 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.079219 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.079381 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.079614 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.079778 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.079929 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.080079 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.080239 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.080396 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.080545 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.080710 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.080875 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.081035 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.081198 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.081355 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.081504 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.081650 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.081811 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.081977 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.082134 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.082283 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.082430 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.082577 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.082732 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.082904 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.083053 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.083201 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.083367 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.083610 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.083764 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.083925 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.084083 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.084233 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.084380 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.084527 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.084726 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.084893 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.085061 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.085236 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.085456 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.085617 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.085824 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.085980 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.086136 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.086285 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.086440 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.086597 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.086759 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.086914 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.087073 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.087226 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.087377 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.087531 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.087770 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.087925 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.088074 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.088229 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.088385 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.088549 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.088715 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.088879 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.089035 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.089200 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.089357 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.089515 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.089746 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.089909 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.090058 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.090213 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.090407 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.090576 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.090736 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.090892 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.091040 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.091204 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.091360 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.091521 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.091677 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.091824 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.091971 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.092125 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.092881 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.093048 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.093206 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.093356 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.093511 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.093771 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.093948 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.094098 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.094252 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.094409 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.094558 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.094716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.094877 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.095026 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.095173 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.095328 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.095492 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.095718 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.095929 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.096096 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.096244 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.096391 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.096539 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.096718 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.096874 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.097022 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.097169 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.097323 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.097486 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.097647 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.097896 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.098060 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.098219 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.098404 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.098560 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.098722 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.098878 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.099034 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.099191 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.099346 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.099502 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.099660 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.099889 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.100045 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.100212 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.100369 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.100528 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.100742 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.100908 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.100908 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.101199 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.101348 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.101505 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.101652 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.101807 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.101954 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.102102 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.102269 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.102633 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.102827 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.102980 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.103137 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.103287 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.103441 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.103604 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.103772 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.104008 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.104163 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.104311 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.104469 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.104631 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.104813 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.104965 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.105111 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.105268 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.105427 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.105576 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.105748 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.106011 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.106176 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.106334 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.106493 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.106661 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.106810 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.106958 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.107113 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.107272 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.107422 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.107580 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.107746 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.107959 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.108735 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.108909 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.109059 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.109218 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.109368 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.109525 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.109681 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.109840 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.110082 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.110237 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.110393 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.110554 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.110719 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.110869 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.111025 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.111174 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.111340 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.111492 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.111654 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.111804 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.111952 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.112107 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.112271 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.112428 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.112428 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.112747 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.112914 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.113066 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.113233 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.113398 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.113548 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.113696 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.113696 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.113977 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.114305 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.114305 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.114615 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.114782 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=118351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 08 02:24:34.114942 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.114942 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.114942 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.114942 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.114942 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.114942 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.114942 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.114942 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.116268 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.116268 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.116268 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.116268 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.116268 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.116268 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.116268 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.116268 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.117630 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.117630 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.117630 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.117630 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.117630 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.117630 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.117630 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.117630 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.118880 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.118880 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.118880 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.118880 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.118880 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.118880 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.118880 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.119899 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.119899 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.120263 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.120415 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.120861 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.121032 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.121330 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.121678 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.122019 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.122343 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.122675 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.122994 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.123323 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.123676 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.124007 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.124455 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.124724 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.125000 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.125342 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.125786 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.125962 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.126332 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.126578 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.126895 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.127214 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.127547 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.127880 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.128195 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.128528 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.128943 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.129546 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.129873 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.130195 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.130536 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.130856 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.131213 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.131480 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.131799 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.132115 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.132438 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.132887 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.133099 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.133421 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.133745 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.134083 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.134418 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.134739 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.135059 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.135390 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.135751 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.136072 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.136473 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.136789 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.137114 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.137437 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.137747 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:24:34.137998 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=118352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 08 02:24:34.265559 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO dbcounter [None req-49c4e202-fae3-40c6-80f1-bc6e611653ee None None] Registered counter for database nova_api Aug 08 02:24:34.286380 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_db.sqlalchemy.engines [None req-49c4e202-fae3-40c6-80f1-bc6e611653ee None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=118351) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 08 02:24:34.292199 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO dbcounter [None req-626911ac-4b3f-4947-9c3a-f69f95993a27 None None] Registered counter for database nova_api Aug 08 02:24:34.293309 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writer thread running {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 08 02:24:34.312276 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_db.sqlalchemy.engines [None req-626911ac-4b3f-4947-9c3a-f69f95993a27 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=118352) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 08 02:24:34.319079 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writer thread running {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 08 02:24:34.351115 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.context [None req-49c4e202-fae3-40c6-80f1-bc6e611653ee None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),3f4309de-b8ab-46d7-9a8f-20127e13d27a(cell1) {{(pid=118351) load_cells /opt/stack/nova/nova/context.py:464}} Aug 08 02:24:34.355265 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-49c4e202-fae3-40c6-80f1-bc6e611653ee None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:24:34.355759 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-49c4e202-fae3-40c6-80f1-bc6e611653ee None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:24:34.357383 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-49c4e202-fae3-40c6-80f1-bc6e611653ee None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:24:34.358122 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-49c4e202-fae3-40c6-80f1-bc6e611653ee None None] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:24:34.358567 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-49c4e202-fae3-40c6-80f1-bc6e611653ee None None] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:24:34.360545 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-49c4e202-fae3-40c6-80f1-bc6e611653ee None None] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:24:34.362620 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO dbcounter [None req-49c4e202-fae3-40c6-80f1-bc6e611653ee None None] Registered counter for database nova_cell0 Aug 08 02:24:34.366761 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO dbcounter [None req-49c4e202-fae3-40c6-80f1-bc6e611653ee None None] Registered counter for database nova_cell1 Aug 08 02:24:34.374164 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_db.sqlalchemy.engines [None req-49c4e202-fae3-40c6-80f1-bc6e611653ee None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=118351) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 08 02:24:34.376085 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_db.sqlalchemy.engines [None req-49c4e202-fae3-40c6-80f1-bc6e611653ee None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=118351) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 08 02:24:34.376580 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.context [None req-626911ac-4b3f-4947-9c3a-f69f95993a27 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),3f4309de-b8ab-46d7-9a8f-20127e13d27a(cell1) {{(pid=118352) load_cells /opt/stack/nova/nova/context.py:464}} Aug 08 02:24:34.380742 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-626911ac-4b3f-4947-9c3a-f69f95993a27 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:24:34.381223 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-626911ac-4b3f-4947-9c3a-f69f95993a27 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:24:34.381643 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writer thread running {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 08 02:24:34.382799 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-626911ac-4b3f-4947-9c3a-f69f95993a27 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:24:34.383535 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-626911ac-4b3f-4947-9c3a-f69f95993a27 None None] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:24:34.383997 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-626911ac-4b3f-4947-9c3a-f69f95993a27 None None] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:24:34.384159 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writer thread running {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 08 02:24:34.386027 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-626911ac-4b3f-4947-9c3a-f69f95993a27 None None] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:24:34.388150 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO dbcounter [None req-626911ac-4b3f-4947-9c3a-f69f95993a27 None None] Registered counter for database nova_cell0 Aug 08 02:24:34.392902 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO dbcounter [None req-626911ac-4b3f-4947-9c3a-f69f95993a27 None None] Registered counter for database nova_cell1 Aug 08 02:24:34.400391 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_db.sqlalchemy.engines [None req-626911ac-4b3f-4947-9c3a-f69f95993a27 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=118352) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 08 02:24:34.402176 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_db.sqlalchemy.engines [None req-626911ac-4b3f-4947-9c3a-f69f95993a27 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=118352) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 08 02:24:34.409560 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writer thread running {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 08 02:24:34.410870 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writer thread running {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 08 02:24:34.529549 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO dbcounter [None req-49c4e202-fae3-40c6-80f1-bc6e611653ee None None] Registered counter for database nova_cell0 Aug 08 02:24:34.536964 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_db.sqlalchemy.engines [None req-49c4e202-fae3-40c6-80f1-bc6e611653ee None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=118351) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 08 02:24:34.541250 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writer thread running {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 08 02:24:34.559601 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO dbcounter [None req-626911ac-4b3f-4947-9c3a-f69f95993a27 None None] Registered counter for database nova_cell0 Aug 08 02:24:34.566983 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_db.sqlalchemy.engines [None req-626911ac-4b3f-4947-9c3a-f69f95993a27 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=118352) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 08 02:24:34.571557 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writer thread running {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 08 02:24:34.859819 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: WARNING keystonemiddleware.auth_token [None req-49c4e202-fae3-40c6-80f1-bc6e611653ee None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Aug 08 02:24:34.867109 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: WARNING keystonemiddleware.auth_token [None req-49c4e202-fae3-40c6-80f1-bc6e611653ee None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Aug 08 02:24:34.890253 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: WARNING keystonemiddleware.auth_token [None req-626911ac-4b3f-4947-9c3a-f69f95993a27 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Aug 08 02:24:34.897641 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: WARNING keystonemiddleware.auth_token [None req-626911ac-4b3f-4947-9c3a-f69f95993a27 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Aug 08 02:24:35.208358 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: WARNING keystonemiddleware.auth_token [None req-49c4e202-fae3-40c6-80f1-bc6e611653ee None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Aug 08 02:24:35.214343 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: WARNING keystonemiddleware.auth_token [None req-49c4e202-fae3-40c6-80f1-bc6e611653ee None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Aug 08 02:24:35.220630 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55d63d2e19c0 pid: 118351 (default app) Aug 08 02:24:35.238625 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: WARNING keystonemiddleware.auth_token [None req-626911ac-4b3f-4947-9c3a-f69f95993a27 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Aug 08 02:24:35.244561 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: WARNING keystonemiddleware.auth_token [None req-626911ac-4b3f-4947-9c3a-f69f95993a27 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Aug 08 02:24:35.251452 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55d63d2e19c0 pid: 118352 (default app) Aug 08 02:25:16.979509 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-361c6426-e4e6-40a9-9ca9-a46a41798cc1 tempest-VolumesTestJSON-738744479 tempest-VolumesTestJSON-738744479-project] Calling method '.version_select at 0x7efc75a5d940>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:17.026950 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_api:SELECT=3 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:17.028157 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell1:SELECT=3 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:17.030150 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell0:SELECT=3 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:17.031009 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell0:SELECT=3 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:17.926605 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-361c6426-e4e6-40a9-9ca9-a46a41798cc1 tempest-VolumesTestJSON-738744479 tempest-VolumesTestJSON-738744479-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/b23f1eb6-9435-43b1-a891-9033e3866cb3" status: 200 len: 351 microversion: 2.1 time: 1.397225 Aug 08 02:25:17.926605 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 1/1] 10.222.0.27 () {58 vars in 1245 bytes} [Mon Aug 8 02:25:16 2022] GET /compute/v2.1/os-volumes/b23f1eb6-9435-43b1-a891-9033e3866cb3 => generated 351 bytes in 1400 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:25:20.324191 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_api:SELECT=3 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:20.325901 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell0:SELECT=3 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:20.337340 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell1:SELECT=3 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:20.338231 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell0:SELECT=3 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:20.650524 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-569e68d8-75fd-403e-99e9-7d61dc58a34f tempest-VolumesTestJSON-738744479 tempest-VolumesTestJSON-738744479-project] Calling method '.version_select at 0x7efc759f48b0>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:21.286677 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-569e68d8-75fd-403e-99e9-7d61dc58a34f tempest-VolumesTestJSON-738744479 tempest-VolumesTestJSON-738744479-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/c07336e3-10e3-41da-bcc5-27dab7bdadde" status: 200 len: 351 microversion: 2.1 time: 0.967725 Aug 08 02:25:21.287272 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 1/2] 10.222.0.27 () {58 vars in 1245 bytes} [Mon Aug 8 02:25:20 2022] GET /compute/v2.1/os-volumes/c07336e3-10e3-41da-bcc5-27dab7bdadde => generated 351 bytes in 970 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:25:23.521128 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-4aaafdfe-32cb-4b26-844a-014118a0dcf2 tempest-VolumesTestJSON-738744479 tempest-VolumesTestJSON-738744479-project] Calling method '.version_select at 0x7efc758afdc0>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:23.646305 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-4aaafdfe-32cb-4b26-844a-014118a0dcf2 tempest-VolumesTestJSON-738744479 tempest-VolumesTestJSON-738744479-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/5c42aa31-64bd-49d3-af86-a38c4698795f" status: 200 len: 351 microversion: 2.1 time: 0.129890 Aug 08 02:25:23.647032 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 2/3] 10.222.0.27 () {58 vars in 1245 bytes} [Mon Aug 8 02:25:23 2022] GET /compute/v2.1/os-volumes/5c42aa31-64bd-49d3-af86-a38c4698795f => generated 351 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:25:23.660993 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-aae42c66-e71f-44a0-a19c-e1adc1b8ce50 tempest-VolumesTestJSON-738744479 tempest-VolumesTestJSON-738744479-project] Calling method '.version_select at 0x7efc75839e50>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:23.781315 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-aae42c66-e71f-44a0-a19c-e1adc1b8ce50 tempest-VolumesTestJSON-738744479 tempest-VolumesTestJSON-738744479-project] 10.222.0.27 "GET /compute/v2.1/os-volumes" status: 200 len: 1036 microversion: 2.1 time: 0.124325 Aug 08 02:25:23.782454 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 2/4] 10.222.0.27 () {58 vars in 1134 bytes} [Mon Aug 8 02:25:23 2022] GET /compute/v2.1/os-volumes => generated 1036 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:23.799547 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-4a26f314-b204-41f9-a1e1-a71ffb664c46 tempest-VolumesTestJSON-738744479 tempest-VolumesTestJSON-738744479-project] Calling method '.version_select at 0x7efc758afb80>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:24.192192 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-4a26f314-b204-41f9-a1e1-a71ffb664c46 tempest-VolumesTestJSON-738744479 tempest-VolumesTestJSON-738744479-project] 10.222.0.27 "GET /compute/v2.1/os-volumes?limit=2" status: 200 len: 695 microversion: 2.1 time: 0.399096 Aug 08 02:25:24.194435 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 3/5] 10.222.0.27 () {58 vars in 1149 bytes} [Mon Aug 8 02:25:23 2022] GET /compute/v2.1/os-volumes?limit=2 => generated 695 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:25:24.219376 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-efbcedcb-1d9e-4bd3-a081-1ae7c17346b7 tempest-VolumesTestJSON-738744479 tempest-VolumesTestJSON-738744479-project] Calling method '.version_select at 0x7efc75839c10>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:24.323644 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-efbcedcb-1d9e-4bd3-a081-1ae7c17346b7 tempest-VolumesTestJSON-738744479 tempest-VolumesTestJSON-738744479-project] 10.222.0.27 "GET /compute/v2.1/os-volumes" status: 200 len: 1036 microversion: 2.1 time: 0.113062 Aug 08 02:25:24.332837 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 3/6] 10.222.0.27 () {58 vars in 1134 bytes} [Mon Aug 8 02:25:24 2022] GET /compute/v2.1/os-volumes => generated 1036 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:24.337797 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-d7f2a975-7162-49cc-bfc4-5178abccbe2c tempest-VolumesTestJSON-738744479 tempest-VolumesTestJSON-738744479-project] Calling method '.version_select at 0x7efc758af790>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:24.519787 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-d7f2a975-7162-49cc-bfc4-5178abccbe2c tempest-VolumesTestJSON-738744479 tempest-VolumesTestJSON-738744479-project] 10.222.0.27 "GET /compute/v2.1/os-volumes?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.185947 Aug 08 02:25:24.530074 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 4/7] 10.222.0.27 () {58 vars in 1167 bytes} [Mon Aug 8 02:25:24 2022] GET /compute/v2.1/os-volumes?offset=1&limit=1 => generated 354 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:25:24.537412 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-d5f2fe1d-1c99-49de-89c9-d5b3b8e4d9cf tempest-VolumesTestJSON-738744479 tempest-VolumesTestJSON-738744479-project] Calling method '.version_select at 0x7efc75811c10>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:24.709414 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-d5f2fe1d-1c99-49de-89c9-d5b3b8e4d9cf tempest-VolumesTestJSON-738744479 tempest-VolumesTestJSON-738744479-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/detail?limit=2" status: 200 len: 695 microversion: 2.1 time: 0.175763 Aug 08 02:25:24.721279 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 4/8] 10.222.0.27 () {58 vars in 1170 bytes} [Mon Aug 8 02:25:24 2022] GET /compute/v2.1/os-volumes/detail?limit=2 => generated 695 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:25:24.726045 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-69490c73-d4e8-4fa0-ac98-6bdb789c9504 tempest-VolumesTestJSON-738744479 tempest-VolumesTestJSON-738744479-project] Calling method '.version_select at 0x7efc75875ee0>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:24.909646 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-69490c73-d4e8-4fa0-ac98-6bdb789c9504 tempest-VolumesTestJSON-738744479 tempest-VolumesTestJSON-738744479-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1036 microversion: 2.1 time: 0.187236 Aug 08 02:25:24.910741 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 5/9] 10.222.0.27 () {58 vars in 1155 bytes} [Mon Aug 8 02:25:24 2022] GET /compute/v2.1/os-volumes/detail => generated 1036 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:24.923660 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-1aa7516c-7495-4305-ad53-c88167526be3 tempest-VolumesTestJSON-738744479 tempest-VolumesTestJSON-738744479-project] Calling method '.version_select at 0x7efc75811a60>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:25.101026 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-1aa7516c-7495-4305-ad53-c88167526be3 tempest-VolumesTestJSON-738744479 tempest-VolumesTestJSON-738744479-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/detail?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.181003 Aug 08 02:25:25.112073 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 5/10] 10.222.0.27 () {58 vars in 1188 bytes} [Mon Aug 8 02:25:24 2022] GET /compute/v2.1/os-volumes/detail?offset=1&limit=1 => generated 354 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:25:25.117850 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-ac0e7c6e-ef2e-4ad7-98e6-83c7c8637c05 tempest-VolumesTestJSON-738744479 tempest-VolumesTestJSON-738744479-project] Calling method '.version_select at 0x7efc75817160>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:25.341793 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-ac0e7c6e-ef2e-4ad7-98e6-83c7c8637c05 tempest-VolumesTestJSON-738744479 tempest-VolumesTestJSON-738744479-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1036 microversion: 2.1 time: 0.227663 Aug 08 02:25:25.342761 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 6/11] 10.222.0.27 () {58 vars in 1155 bytes} [Mon Aug 8 02:25:25 2022] GET /compute/v2.1/os-volumes/detail => generated 1036 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:25.535980 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-936b8ee1-af17-424d-8b12-031c2488bcb6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Action: 'create', calling method: .version_select at 0x7efc76187160>, body: {"keypair": {"name": "tempest-keypair-1410729274", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBC10mDPIm0ojkapOG+RMdDjML69iBKKXmoP0I4SrBwOSpQDStnv9pBpQmDULxXUGfuzSvWI/njYecMQtP05SJAtndk5RCD4rQYOcHYxZM0o3TU9a9xg3S7dlhJWqsZNEAQ=="}} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:25:25.649863 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.quota [None req-936b8ee1-af17-424d-8b12-031c2488bcb6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Getting quotas for project ca29b2b9a59d4826bfd25f4dcfa9cffe. Resources: {'key_pairs'} {{(pid=118352) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 08 02:25:25.671471 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.quota [None req-936b8ee1-af17-424d-8b12-031c2488bcb6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Getting quotas for user 578cd837d9fe442587f8c6c8528334c0 and project ca29b2b9a59d4826bfd25f4dcfa9cffe. Resources: {'key_pairs'} {{(pid=118352) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 08 02:25:25.906416 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-936b8ee1-af17-424d-8b12-031c2488bcb6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.741116 Aug 08 02:25:25.909321 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 6/12] 10.222.0.27 () {60 vars in 1159 bytes} [Mon Aug 8 02:25:25 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 745 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:25:28.121570 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-9c0f4632-d73a-4823-b620-20fe189ed3a5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1029702361", "imageRef": "30833f47-5987-41ca-b5ee-ad5ecafbb265", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:25:28.333114 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.network.neutron [None req-9c0f4632-d73a-4823-b620-20fe189ed3a5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] validate_networks() for [('847f3ad1-eef7-41cf-8570-7f9863393002', None, None, None, None, None)] {{(pid=118351) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 08 02:25:29.992989 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-ab8849ee-b55b-4797-9c28-51a0e99d8669 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Action: 'create', calling method: .version_select at 0x7efc757ec940>, body: {"keypair": {"name": "tempest-keypair-1673745916", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBEpPXrkbrgBq2f3ttycVT+yk2FhlE6SPzjcSTi+SzmRmiksCLaneafZ7yQ+Ga+XWQoI2rX/4U3hoe83dohT/2JYXlk6Cusk+RTDAIhrHT/pTMxGLQJR0o8AlM2f+o5BYQ=="}} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:25:30.036368 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.quota [None req-ab8849ee-b55b-4797-9c28-51a0e99d8669 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Getting quotas for project 7caa43bf43ad4e84b4b648e9ab33bd7f. Resources: {'key_pairs'} {{(pid=118352) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 08 02:25:30.047804 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.quota [None req-ab8849ee-b55b-4797-9c28-51a0e99d8669 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Getting quotas for user d451fb2f26ce4f50b010fb2a387c4bf3 and project 7caa43bf43ad4e84b4b648e9ab33bd7f. Resources: {'key_pairs'} {{(pid=118352) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 08 02:25:30.243973 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-ab8849ee-b55b-4797-9c28-51a0e99d8669 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.20 time: 0.368147 Aug 08 02:25:30.253927 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 7/13] 10.222.0.27 () {62 vars in 1200 bytes} [Mon Aug 8 02:25:29 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 379 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Aug 08 02:25:30.794588 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-c5169c96-9c37-48c1-bd9a-2bdb1451f096 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Action: 'create', calling method: .version_select at 0x7efc757a4430>, body: {"keypair": {"name": "tempest-keypair-2085665456", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKuX1337+crUGYhDo4Wu+cyvoLQZVyfL9b4NYHn6s4VaaOAHicCyyHBwXM+OjEjmSyv3hyC9ro4E1mUMqbjGr+ppMJhvKsDOGxdHUTR8+FSccTRPsRxRt3JWp1HqbYMoNg=="}} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:25:30.845291 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.quota [None req-c5169c96-9c37-48c1-bd9a-2bdb1451f096 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Getting quotas for project ee8baa7d6e294e27ae2fb12eded3709d. Resources: {'key_pairs'} {{(pid=118352) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 08 02:25:30.859977 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.quota [None req-c5169c96-9c37-48c1-bd9a-2bdb1451f096 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Getting quotas for user eb4c92edcb024e32a549650eb85f6952 and project ee8baa7d6e294e27ae2fb12eded3709d. Resources: {'key_pairs'} {{(pid=118352) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 08 02:25:31.177639 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-c5169c96-9c37-48c1-bd9a-2bdb1451f096 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.495178 Aug 08 02:25:31.188643 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 8/14] 10.222.0.27 () {60 vars in 1159 bytes} [Mon Aug 8 02:25:30 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 508 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:25:32.239610 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.virt.hardware [None req-9c0f4632-d73a-4823-b620-20fe189ed3a5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Flavor limits 0:0:0 {{(pid=118351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 08 02:25:32.240144 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.virt.hardware [None req-9c0f4632-d73a-4823-b620-20fe189ed3a5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Image limits 0:0:0 {{(pid=118351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 08 02:25:32.240490 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.virt.hardware [None req-9c0f4632-d73a-4823-b620-20fe189ed3a5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Flavor pref 0:0:0 {{(pid=118351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 08 02:25:32.240929 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.virt.hardware [None req-9c0f4632-d73a-4823-b620-20fe189ed3a5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Image pref 0:0:0 {{(pid=118351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 08 02:25:32.241350 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.virt.hardware [None req-9c0f4632-d73a-4823-b620-20fe189ed3a5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 08 02:25:32.263184 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-9c0f4632-d73a-4823-b620-20fe189ed3a5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:32.263741 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-9c0f4632-d73a-4823-b620-20fe189ed3a5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:32.264537 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-9c0f4632-d73a-4823-b620-20fe189ed3a5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:32.267250 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-9c0f4632-d73a-4823-b620-20fe189ed3a5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:32.269140 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-9c0f4632-d73a-4823-b620-20fe189ed3a5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:32.269982 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-9c0f4632-d73a-4823-b620-20fe189ed3a5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:32.319594 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.quota [None req-9c0f4632-d73a-4823-b620-20fe189ed3a5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Getting quotas for project 8af7d2bbc6d94273bd39c81423d8cebe. Resources: {'ram', 'cores', 'instances'} {{(pid=118351) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 08 02:25:32.356371 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.quota [None req-9c0f4632-d73a-4823-b620-20fe189ed3a5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Getting quotas for user 5c8ae69712c14878bd2aab6d33512b7a and project 8af7d2bbc6d94273bd39c81423d8cebe. Resources: {'ram', 'cores', 'instances'} {{(pid=118351) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 08 02:25:32.393676 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-9c0f4632-d73a-4823-b620-20fe189ed3a5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Going to run 1 instances... {{(pid=118351) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Aug 08 02:25:32.401351 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-9c0f4632-d73a-4823-b620-20fe189ed3a5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: ed75f58f-9135-4d97-b24a-2f8fcdab53ce] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='30833f47-5987-41ca-b5ee-ad5ecafbb265',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118351) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Aug 08 02:25:32.748987 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-9c0f4632-d73a-4823-b620-20fe189ed3a5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 4.853432 Aug 08 02:25:32.757797 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 7/15] 10.222.0.27 () {60 vars in 1147 bytes} [Mon Aug 8 02:25:27 2022] POST /compute/v2.1/servers => generated 383 bytes in 4863 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 08 02:25:32.761870 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-300066bd-716e-4c10-8e8d-2c110df742d9 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:32.764563 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-300066bd-716e-4c10-8e8d-2c110df742d9 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: ed75f58f-9135-4d97-b24a-2f8fcdab53ce] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:32.961441 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-300066bd-716e-4c10-8e8d-2c110df742d9 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:32.998175 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-300066bd-716e-4c10-8e8d-2c110df742d9 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce" status: 200 len: 1222 microversion: 2.1 time: 0.239576 Aug 08 02:25:32.999404 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 9/16] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:32 2022] GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce => generated 1222 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:33.209754 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-1ef9457b-087f-4a6f-910d-22e9eb526731 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1043423033", "imageRef": "30833f47-5987-41ca-b5ee-ad5ecafbb265", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:25:33.313814 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.network.neutron [None req-1ef9457b-087f-4a6f-910d-22e9eb526731 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] validate_networks() for [('847f3ad1-eef7-41cf-8570-7f9863393002', None, None, None, None, None)] {{(pid=118351) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 08 02:25:34.662406 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.virt.hardware [None req-1ef9457b-087f-4a6f-910d-22e9eb526731 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Flavor limits 0:0:0 {{(pid=118351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 08 02:25:34.662987 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.virt.hardware [None req-1ef9457b-087f-4a6f-910d-22e9eb526731 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Image limits 0:0:0 {{(pid=118351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 08 02:25:34.663300 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.virt.hardware [None req-1ef9457b-087f-4a6f-910d-22e9eb526731 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Flavor pref 0:0:0 {{(pid=118351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 08 02:25:34.663711 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.virt.hardware [None req-1ef9457b-087f-4a6f-910d-22e9eb526731 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Image pref 0:0:0 {{(pid=118351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 08 02:25:34.664132 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.virt.hardware [None req-1ef9457b-087f-4a6f-910d-22e9eb526731 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 08 02:25:34.702353 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1ef9457b-087f-4a6f-910d-22e9eb526731 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:34.702353 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1ef9457b-087f-4a6f-910d-22e9eb526731 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:34.702353 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1ef9457b-087f-4a6f-910d-22e9eb526731 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:34.705339 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1ef9457b-087f-4a6f-910d-22e9eb526731 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:34.705967 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1ef9457b-087f-4a6f-910d-22e9eb526731 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:34.706721 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1ef9457b-087f-4a6f-910d-22e9eb526731 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:34.767533 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.quota [None req-1ef9457b-087f-4a6f-910d-22e9eb526731 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Getting quotas for project 8af7d2bbc6d94273bd39c81423d8cebe. Resources: {'ram', 'cores', 'instances'} {{(pid=118351) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 08 02:25:34.849607 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.quota [None req-1ef9457b-087f-4a6f-910d-22e9eb526731 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Getting quotas for user 5c8ae69712c14878bd2aab6d33512b7a and project 8af7d2bbc6d94273bd39c81423d8cebe. Resources: {'ram', 'cores', 'instances'} {{(pid=118351) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 08 02:25:34.849607 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-1ef9457b-087f-4a6f-910d-22e9eb526731 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Going to run 1 instances... {{(pid=118351) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Aug 08 02:25:34.849607 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-1ef9457b-087f-4a6f-910d-22e9eb526731 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='30833f47-5987-41ca-b5ee-ad5ecafbb265',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118351) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Aug 08 02:25:35.079784 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-1ef9457b-087f-4a6f-910d-22e9eb526731 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 1.829283 Aug 08 02:25:35.079784 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 8/17] 10.222.0.27 () {60 vars in 1147 bytes} [Mon Aug 8 02:25:33 2022] POST /compute/v2.1/servers => generated 383 bytes in 1831 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 08 02:25:35.079784 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-832afab3-b4eb-4f0d-a4e6-95abbefcd487 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:35.079784 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-832afab3-b4eb-4f0d-a4e6-95abbefcd487 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:35.129974 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-832afab3-b4eb-4f0d-a4e6-95abbefcd487 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:35.147152 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-832afab3-b4eb-4f0d-a4e6-95abbefcd487 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1222 microversion: 2.1 time: 0.105124 Aug 08 02:25:35.147936 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 9/18] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:35 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1222 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:35.752188 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-82d08b77-9be2-4e4c-89cf-f72e519212eb tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-943657222", "imageRef": "30833f47-5987-41ca-b5ee-ad5ecafbb265", "flavorRef": "452", "networks": [{"uuid": "beb9e56d-e621-4ebe-bc3c-f9e5f759c12a"}]}} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:25:36.017450 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-133d591c-8545-4705-8bea-910c3dead87e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-797044633", "imageRef": "30833f47-5987-41ca-b5ee-ad5ecafbb265", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "f92c936a-6950-4530-8001-5260ea91adac"}], "security_groups": [{"name": "tempest-securitygroup--911812410"}], "key_name": "tempest-keypair-1410729274", "user_data": "}} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:25:36.023438 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.network.neutron [None req-82d08b77-9be2-4e4c-89cf-f72e519212eb tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] validate_networks() for [('beb9e56d-e621-4ebe-bc3c-f9e5f759c12a', None, None, None, None, None)] {{(pid=118352) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 08 02:25:36.389591 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.network.neutron [None req-133d591c-8545-4705-8bea-910c3dead87e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] validate_networks() for [('f92c936a-6950-4530-8001-5260ea91adac', None, None, None, None, None)] {{(pid=118351) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 08 02:25:37.954429 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-82d08b77-9be2-4e4c-89cf-f72e519212eb tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Flavor limits 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 08 02:25:37.955001 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-82d08b77-9be2-4e4c-89cf-f72e519212eb tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Image limits 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 08 02:25:37.955666 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-82d08b77-9be2-4e4c-89cf-f72e519212eb tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Flavor pref 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 08 02:25:37.956434 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-82d08b77-9be2-4e4c-89cf-f72e519212eb tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Image pref 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 08 02:25:37.957074 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-82d08b77-9be2-4e4c-89cf-f72e519212eb tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 08 02:25:37.957792 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.virt.hardware [None req-133d591c-8545-4705-8bea-910c3dead87e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Flavor limits 0:0:0 {{(pid=118351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 08 02:25:37.958310 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.virt.hardware [None req-133d591c-8545-4705-8bea-910c3dead87e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Image limits 0:0:0 {{(pid=118351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 08 02:25:37.958714 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.virt.hardware [None req-133d591c-8545-4705-8bea-910c3dead87e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Flavor pref 0:0:0 {{(pid=118351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 08 02:25:37.959146 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.virt.hardware [None req-133d591c-8545-4705-8bea-910c3dead87e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Image pref 0:0:0 {{(pid=118351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 08 02:25:37.959868 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.virt.hardware [None req-133d591c-8545-4705-8bea-910c3dead87e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 08 02:25:37.975236 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-133d591c-8545-4705-8bea-910c3dead87e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:37.976649 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-133d591c-8545-4705-8bea-910c3dead87e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:37.977569 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-133d591c-8545-4705-8bea-910c3dead87e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:37.982795 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-133d591c-8545-4705-8bea-910c3dead87e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:37.984605 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-133d591c-8545-4705-8bea-910c3dead87e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:37.987068 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-133d591c-8545-4705-8bea-910c3dead87e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:37.989441 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-82d08b77-9be2-4e4c-89cf-f72e519212eb tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:37.991696 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-82d08b77-9be2-4e4c-89cf-f72e519212eb tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:37.996927 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-82d08b77-9be2-4e4c-89cf-f72e519212eb tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:37.998824 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-82d08b77-9be2-4e4c-89cf-f72e519212eb tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:37.999591 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-82d08b77-9be2-4e4c-89cf-f72e519212eb tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:38.000331 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-82d08b77-9be2-4e4c-89cf-f72e519212eb tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:38.027653 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.quota [None req-133d591c-8545-4705-8bea-910c3dead87e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Getting quotas for project ca29b2b9a59d4826bfd25f4dcfa9cffe. Resources: {'ram', 'cores', 'instances'} {{(pid=118351) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 08 02:25:38.034112 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.quota [None req-133d591c-8545-4705-8bea-910c3dead87e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Getting quotas for user 578cd837d9fe442587f8c6c8528334c0 and project ca29b2b9a59d4826bfd25f4dcfa9cffe. Resources: {'ram', 'cores', 'instances'} {{(pid=118351) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 08 02:25:38.037397 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.quota [None req-82d08b77-9be2-4e4c-89cf-f72e519212eb tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Getting quotas for project 5515114a058944a49182063213de54d3. Resources: {'ram', 'cores', 'instances'} {{(pid=118352) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 08 02:25:38.043451 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.quota [None req-82d08b77-9be2-4e4c-89cf-f72e519212eb tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Getting quotas for user 2aeb52d51d3f44319bb4e114858222f0 and project 5515114a058944a49182063213de54d3. Resources: {'ram', 'cores', 'instances'} {{(pid=118352) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 08 02:25:38.049458 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-133d591c-8545-4705-8bea-910c3dead87e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Going to run 1 instances... {{(pid=118351) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Aug 08 02:25:38.051926 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-133d591c-8545-4705-8bea-910c3dead87e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: 95ae6b45-35d7-43a7-a021-91bb545cac4e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='30833f47-5987-41ca-b5ee-ad5ecafbb265',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118351) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Aug 08 02:25:38.057750 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-82d08b77-9be2-4e4c-89cf-f72e519212eb tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Going to run 1 instances... {{(pid=118352) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Aug 08 02:25:38.062184 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-82d08b77-9be2-4e4c-89cf-f72e519212eb tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] [instance: 0fc3bfe1-b902-4a65-8c41-81e6f49112ae] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='30833f47-5987-41ca-b5ee-ad5ecafbb265',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118352) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Aug 08 02:25:38.245045 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-133d591c-8545-4705-8bea-910c3dead87e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 2.233027 Aug 08 02:25:38.247294 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 10/19] 10.222.0.27 () {60 vars in 1147 bytes} [Mon Aug 8 02:25:36 2022] POST /compute/v2.1/servers => generated 401 bytes in 2236 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 08 02:25:38.268050 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-d81da064-7923-4215-a7fb-5e1b856c6766 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:38.270923 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-d81da064-7923-4215-a7fb-5e1b856c6766 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:38.286286 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d81da064-7923-4215-a7fb-5e1b856c6766 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:38.289105 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d81da064-7923-4215-a7fb-5e1b856c6766 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:38.289980 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d81da064-7923-4215-a7fb-5e1b856c6766 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:38.293104 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d81da064-7923-4215-a7fb-5e1b856c6766 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:38.297006 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d81da064-7923-4215-a7fb-5e1b856c6766 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:38.298446 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d81da064-7923-4215-a7fb-5e1b856c6766 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:38.344279 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-82d08b77-9be2-4e4c-89cf-f72e519212eb tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 3.302415 Aug 08 02:25:38.353488 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 10/20] 10.222.0.27 () {60 vars in 1147 bytes} [Mon Aug 8 02:25:35 2022] POST /compute/v2.1/servers => generated 380 bytes in 3312 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 08 02:25:38.715347 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-d81da064-7923-4215-a7fb-5e1b856c6766 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:38.767619 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-d81da064-7923-4215-a7fb-5e1b856c6766 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1274 microversion: 2.1 time: 0.508327 Aug 08 02:25:38.768061 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 11/21] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:38 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1274 bytes in 516 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:38.822934 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-f3f2d311-7ea3-4c81-abc4-ceedc933c30f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-1610696250", "imageRef": "30833f47-5987-41ca-b5ee-ad5ecafbb265", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "637774ba-5318-4e0b-8885-18933b06fb53"}], "security_groups": [{"name": "tempest-securitygroup--1339761804"}], "key_name": "tempest-keypair-1673745916", "user_data": "}} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:25:38.846196 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-2cceafff-08e5-4c63-bed6-22f6a12df1da tempest-VolumesNegativeTest-1884126584 tempest-VolumesNegativeTest-1884126584-project] Action: 'create', calling method: .version_select at 0x7efc74ed8ee0>, body: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-1304321657", "metadata": {"Type": "work"}}} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:25:38.863406 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-2cceafff-08e5-4c63-bed6-22f6a12df1da tempest-VolumesNegativeTest-1884126584 tempest-VolumesNegativeTest-1884126584-project] Request method failure captured: Aug 08 02:25:38.863406 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Aug 08 02:25:38.863406 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept: application/json Aug 08 02:25:38.863406 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept-Encoding: identity Aug 08 02:25:38.863406 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Connection: close Aug 08 02:25:38.863406 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Length: 122 Aug 08 02:25:38.863406 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Type: application/json Aug 08 02:25:38.863406 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Host: 10.222.0.27 Aug 08 02:25:38.863406 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Openstack-System-Scope: None Aug 08 02:25:38.863406 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: User-Agent: python-urllib3/1.26.11 Aug 08 02:25:38.863406 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Auth-Token: gAAAAABi8HQgw_1OZEEF-8sA44T9D66ncWMNU0dHE8Alu0u97Euaxv99-3vsK2qgZ4PXcKZnYhK5dIlaxHl8Ki2j-95fV2wYZuK5Sljh_nsGxg855cyWr8cNIF3KfbYbGg1nZijDmm8kxC5jLHotF57JKf1wBvsr5gLFRRD5TxvTvGkspPSPFMw Aug 08 02:25:38.863406 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Id: None Aug 08 02:25:38.863406 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Name: None Aug 08 02:25:38.863406 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Identity-Status: Confirmed Aug 08 02:25:38.863406 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Is-Admin-Project: True Aug 08 02:25:38.863406 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Id: default Aug 08 02:25:38.863406 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Name: Default Aug 08 02:25:38.863406 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Id: d2f4a8558d8242c88877511889b2d6e5 Aug 08 02:25:38.863406 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Name: tempest-VolumesNegativeTest-1884126584 Aug 08 02:25:38.863406 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Role: reader,member Aug 08 02:25:38.863406 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Roles: reader,member Aug 08 02:25:38.864828 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/d2f4a8558d8242c88877511889b2d6e5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/d2f4a8558d8242c88877511889b2d6e5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_d2f4a8558d8242c88877511889b2d6e5", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/d2f4a8558d8242c88877511889b2d6e5"}]}] Aug 08 02:25:38.864828 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant: tempest-VolumesNegativeTest-1884126584 Aug 08 02:25:38.864828 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Id: d2f4a8558d8242c88877511889b2d6e5 Aug 08 02:25:38.864828 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Name: tempest-VolumesNegativeTest-1884126584 Aug 08 02:25:38.864828 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User: tempest-VolumesNegativeTest-1884126584-project Aug 08 02:25:38.864828 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Id: default Aug 08 02:25:38.864828 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Name: Default Aug 08 02:25:38.864828 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Id: 0925fdcf05e442e3924759a24a3ee099 Aug 08 02:25:38.864828 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Name: tempest-VolumesNegativeTest-1884126584-project Aug 08 02:25:38.864828 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Aug 08 02:25:38.864828 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-1304321657", "metadata": {"Type": "work"}}} Aug 08 02:25:38.864828 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: method: Aug 08 02:25:38.864828 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: exception: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Aug 08 02:25:38.865654 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: action_args: {'body': {'volume': {'size': '#$%', 'display_name': 'tempest-VolumesNegativeTest-Volume-1304321657', 'metadata': {'Type': 'work'}}}} Aug 08 02:25:38.865654 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:25:38.865654 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:25:38.865654 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Aug 08 02:25:38.865654 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Aug 08 02:25:38.865654 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 268, in validate Aug 08 02:25:38.865654 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise error Aug 08 02:25:38.865654 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '#$%' does not match '^[0-9]+$' Aug 08 02:25:38.865654 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:25:38.865654 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Aug 08 02:25:38.865654 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Aug 08 02:25:38.865654 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:25:38.865654 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi On instance['volume']: Aug 08 02:25:38.865654 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi '#$%' Aug 08 02:25:38.865654 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:25:38.865654 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:25:38.865654 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:25:38.865654 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:25:38.865654 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:25:38.865654 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:25:38.865654 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:25:38.865654 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:25:38.866739 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 08 02:25:38.866739 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 08 02:25:38.866739 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:25:38.866739 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:25:38.866739 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Aug 08 02:25:38.866739 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Aug 08 02:25:38.866739 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Aug 08 02:25:38.866739 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Aug 08 02:25:38.866739 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Aug 08 02:25:38.866739 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Aug 08 02:25:38.866739 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Aug 08 02:25:38.866739 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:25:38.866739 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:25:38.866739 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:25:38.866739 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:25:38.866739 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:25:38.866739 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:25:38.866739 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Aug 08 02:25:38.866739 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Aug 08 02:25:38.868746 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Aug 08 02:25:38.868746 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:25:38.868746 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-2cceafff-08e5-4c63-bed6-22f6a12df1da tempest-VolumesNegativeTest-1884126584 tempest-VolumesNegativeTest-1884126584-project] Returning 400 to user: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' {{(pid=118352) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:25:38.868746 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-2cceafff-08e5-4c63-bed6-22f6a12df1da tempest-VolumesNegativeTest-1884126584 tempest-VolumesNegativeTest-1884126584-project] 10.222.0.27 "POST /compute/v2.1/os-volumes" status: 400 len: 127 microversion: 2.1 time: 0.510118 Aug 08 02:25:38.879771 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 11/22] 10.222.0.27 () {60 vars in 1156 bytes} [Mon Aug 8 02:25:38 2022] POST /compute/v2.1/os-volumes => generated 127 bytes in 525 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Aug 08 02:25:38.905519 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-3dff44fb-cc2f-448f-90c9-9290d807cab5 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:38.906251 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-3dff44fb-cc2f-448f-90c9-9290d807cab5 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: 95ae6b45-35d7-43a7-a021-91bb545cac4e] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:39.036986 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-3dff44fb-cc2f-448f-90c9-9290d807cab5 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:39.045593 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-3dff44fb-cc2f-448f-90c9-9290d807cab5 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e" status: 200 len: 1241 microversion: 2.1 time: 0.160545 Aug 08 02:25:39.046489 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 12/23] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:38 2022] GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e => generated 1241 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:39.051723 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-7a55894a-7a5d-4d96-938d-28fc4a6e7e8b tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:39.052334 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-7a55894a-7a5d-4d96-938d-28fc4a6e7e8b tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] [instance: 0fc3bfe1-b902-4a65-8c41-81e6f49112ae] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:39.090036 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.network.neutron [None req-f3f2d311-7ea3-4c81-abc4-ceedc933c30f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] validate_networks() for [('637774ba-5318-4e0b-8885-18933b06fb53', None, None, None, None, None)] {{(pid=118351) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 08 02:25:39.198693 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-7a55894a-7a5d-4d96-938d-28fc4a6e7e8b tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2aeb52d51d3f44319bb4e114858222f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5515114a058944a49182063213de54d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:39.221413 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-7a55894a-7a5d-4d96-938d-28fc4a6e7e8b tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] 10.222.0.27 "GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae" status: 200 len: 1214 microversion: 2.1 time: 0.173737 Aug 08 02:25:39.222017 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 13/24] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:39 2022] GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae => generated 1214 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:39.251893 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-123a374c-3e60-40f3-a671-75c95ce4a29b tempest-VolumesNegativeTest-1884126584 tempest-VolumesNegativeTest-1884126584-project] Action: 'create', calling method: .version_select at 0x7efc74ea0790>, body: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-1657811290", "metadata": {"Type": "work"}}} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:25:39.260007 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-123a374c-3e60-40f3-a671-75c95ce4a29b tempest-VolumesNegativeTest-1884126584 tempest-VolumesNegativeTest-1884126584-project] Request method failure captured: Aug 08 02:25:39.260007 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Aug 08 02:25:39.260007 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept: application/json Aug 08 02:25:39.260007 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept-Encoding: identity Aug 08 02:25:39.260007 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Connection: close Aug 08 02:25:39.260007 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Length: 120 Aug 08 02:25:39.260007 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Type: application/json Aug 08 02:25:39.260007 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Host: 10.222.0.27 Aug 08 02:25:39.260007 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Openstack-System-Scope: None Aug 08 02:25:39.260007 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: User-Agent: python-urllib3/1.26.11 Aug 08 02:25:39.260007 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Auth-Token: gAAAAABi8HQgw_1OZEEF-8sA44T9D66ncWMNU0dHE8Alu0u97Euaxv99-3vsK2qgZ4PXcKZnYhK5dIlaxHl8Ki2j-95fV2wYZuK5Sljh_nsGxg855cyWr8cNIF3KfbYbGg1nZijDmm8kxC5jLHotF57JKf1wBvsr5gLFRRD5TxvTvGkspPSPFMw Aug 08 02:25:39.260007 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Id: None Aug 08 02:25:39.260007 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Name: None Aug 08 02:25:39.260007 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Identity-Status: Confirmed Aug 08 02:25:39.260007 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Is-Admin-Project: True Aug 08 02:25:39.260007 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Id: default Aug 08 02:25:39.260007 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Name: Default Aug 08 02:25:39.260007 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Id: d2f4a8558d8242c88877511889b2d6e5 Aug 08 02:25:39.260007 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Name: tempest-VolumesNegativeTest-1884126584 Aug 08 02:25:39.260007 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Role: reader,member Aug 08 02:25:39.260007 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Roles: reader,member Aug 08 02:25:39.261394 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/d2f4a8558d8242c88877511889b2d6e5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/d2f4a8558d8242c88877511889b2d6e5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_d2f4a8558d8242c88877511889b2d6e5", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/d2f4a8558d8242c88877511889b2d6e5"}]}] Aug 08 02:25:39.261394 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant: tempest-VolumesNegativeTest-1884126584 Aug 08 02:25:39.261394 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Id: d2f4a8558d8242c88877511889b2d6e5 Aug 08 02:25:39.261394 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Name: tempest-VolumesNegativeTest-1884126584 Aug 08 02:25:39.261394 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User: tempest-VolumesNegativeTest-1884126584-project Aug 08 02:25:39.261394 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Id: default Aug 08 02:25:39.261394 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Name: Default Aug 08 02:25:39.261394 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Id: 0925fdcf05e442e3924759a24a3ee099 Aug 08 02:25:39.261394 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Name: tempest-VolumesNegativeTest-1884126584-project Aug 08 02:25:39.261394 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Aug 08 02:25:39.261394 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-1657811290", "metadata": {"Type": "work"}}} Aug 08 02:25:39.261394 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: method: Aug 08 02:25:39.261394 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: exception: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Aug 08 02:25:39.262202 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: action_args: {'body': {'volume': {'size': '0', 'display_name': 'tempest-VolumesNegativeTest-Volume-1657811290', 'metadata': {'Type': 'work'}}}} Aug 08 02:25:39.262202 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:25:39.262202 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:25:39.262202 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Aug 08 02:25:39.262202 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Aug 08 02:25:39.262202 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 268, in validate Aug 08 02:25:39.262202 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise error Aug 08 02:25:39.262202 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 Aug 08 02:25:39.262202 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:25:39.262202 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['volume']['properties']['size']: Aug 08 02:25:39.262202 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Aug 08 02:25:39.262202 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:25:39.262202 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi On instance['volume']: Aug 08 02:25:39.262202 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi '0' Aug 08 02:25:39.262202 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:25:39.262202 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:25:39.262202 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:25:39.262202 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:25:39.262202 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:25:39.262202 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:25:39.262202 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:25:39.262202 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:25:39.263437 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 08 02:25:39.263437 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 08 02:25:39.263437 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:25:39.263437 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:25:39.263437 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Aug 08 02:25:39.263437 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Aug 08 02:25:39.263437 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Aug 08 02:25:39.263437 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Aug 08 02:25:39.263437 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Aug 08 02:25:39.263437 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Aug 08 02:25:39.263437 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Aug 08 02:25:39.263437 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:25:39.263437 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:25:39.263437 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:25:39.263437 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:25:39.263437 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:25:39.263437 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:25:39.263437 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Aug 08 02:25:39.263437 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Aug 08 02:25:39.265834 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Aug 08 02:25:39.265834 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:25:39.267098 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-123a374c-3e60-40f3-a671-75c95ce4a29b tempest-VolumesNegativeTest-1884126584 tempest-VolumesNegativeTest-1884126584-project] Returning 400 to user: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 {{(pid=118352) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:25:39.270516 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-123a374c-3e60-40f3-a671-75c95ce4a29b tempest-VolumesNegativeTest-1884126584 tempest-VolumesNegativeTest-1884126584-project] 10.222.0.27 "POST /compute/v2.1/os-volumes" status: 400 len: 125 microversion: 2.1 time: 0.023079 Aug 08 02:25:39.276450 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 14/25] 10.222.0.27 () {60 vars in 1156 bytes} [Mon Aug 8 02:25:39 2022] POST /compute/v2.1/os-volumes => generated 125 bytes in 54 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Aug 08 02:25:39.288800 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-562af204-7869-414f-9c33-dd13974c22c4 tempest-VolumesNegativeTest-1884126584 tempest-VolumesNegativeTest-1884126584-project] Action: 'create', calling method: .version_select at 0x7efc74ea0ee0>, body: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-342006747", "metadata": {"Type": "work"}}} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:25:39.299291 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-562af204-7869-414f-9c33-dd13974c22c4 tempest-VolumesNegativeTest-1884126584 tempest-VolumesNegativeTest-1884126584-project] Request method failure captured: Aug 08 02:25:39.299291 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Aug 08 02:25:39.299291 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept: application/json Aug 08 02:25:39.299291 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept-Encoding: identity Aug 08 02:25:39.299291 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Connection: close Aug 08 02:25:39.299291 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Length: 118 Aug 08 02:25:39.299291 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Type: application/json Aug 08 02:25:39.299291 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Host: 10.222.0.27 Aug 08 02:25:39.299291 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Openstack-System-Scope: None Aug 08 02:25:39.299291 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: User-Agent: python-urllib3/1.26.11 Aug 08 02:25:39.299291 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Auth-Token: gAAAAABi8HQgw_1OZEEF-8sA44T9D66ncWMNU0dHE8Alu0u97Euaxv99-3vsK2qgZ4PXcKZnYhK5dIlaxHl8Ki2j-95fV2wYZuK5Sljh_nsGxg855cyWr8cNIF3KfbYbGg1nZijDmm8kxC5jLHotF57JKf1wBvsr5gLFRRD5TxvTvGkspPSPFMw Aug 08 02:25:39.299291 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Id: None Aug 08 02:25:39.299291 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Name: None Aug 08 02:25:39.299291 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Identity-Status: Confirmed Aug 08 02:25:39.299291 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Is-Admin-Project: True Aug 08 02:25:39.299291 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Id: default Aug 08 02:25:39.299291 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Name: Default Aug 08 02:25:39.299291 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Id: d2f4a8558d8242c88877511889b2d6e5 Aug 08 02:25:39.299291 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Name: tempest-VolumesNegativeTest-1884126584 Aug 08 02:25:39.299291 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Role: reader,member Aug 08 02:25:39.299291 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Roles: reader,member Aug 08 02:25:39.300324 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/d2f4a8558d8242c88877511889b2d6e5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/d2f4a8558d8242c88877511889b2d6e5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_d2f4a8558d8242c88877511889b2d6e5", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/d2f4a8558d8242c88877511889b2d6e5"}]}] Aug 08 02:25:39.300324 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant: tempest-VolumesNegativeTest-1884126584 Aug 08 02:25:39.300324 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Id: d2f4a8558d8242c88877511889b2d6e5 Aug 08 02:25:39.300324 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Name: tempest-VolumesNegativeTest-1884126584 Aug 08 02:25:39.300324 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User: tempest-VolumesNegativeTest-1884126584-project Aug 08 02:25:39.300324 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Id: default Aug 08 02:25:39.300324 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Name: Default Aug 08 02:25:39.300324 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Id: 0925fdcf05e442e3924759a24a3ee099 Aug 08 02:25:39.300324 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Name: tempest-VolumesNegativeTest-1884126584-project Aug 08 02:25:39.300324 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Aug 08 02:25:39.300324 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-342006747", "metadata": {"Type": "work"}}} Aug 08 02:25:39.300324 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: method: Aug 08 02:25:39.300324 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: exception: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Aug 08 02:25:39.301086 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: action_args: {'body': {'volume': {'size': '', 'display_name': 'tempest-VolumesNegativeTest-Volume-342006747', 'metadata': {'Type': 'work'}}}} Aug 08 02:25:39.301086 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:25:39.301086 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:25:39.301086 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Aug 08 02:25:39.301086 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Aug 08 02:25:39.301086 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 268, in validate Aug 08 02:25:39.301086 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise error Aug 08 02:25:39.301086 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match '^[0-9]+$' Aug 08 02:25:39.301086 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:25:39.301086 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Aug 08 02:25:39.301086 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Aug 08 02:25:39.301086 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:25:39.301086 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi On instance['volume']: Aug 08 02:25:39.301086 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi '' Aug 08 02:25:39.301086 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:25:39.301086 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:25:39.301086 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:25:39.301086 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:25:39.301086 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:25:39.301086 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:25:39.301086 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:25:39.301086 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:25:39.302142 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 08 02:25:39.302142 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 08 02:25:39.302142 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:25:39.302142 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:25:39.302142 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Aug 08 02:25:39.302142 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Aug 08 02:25:39.302142 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Aug 08 02:25:39.302142 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Aug 08 02:25:39.302142 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Aug 08 02:25:39.302142 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Aug 08 02:25:39.302142 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Aug 08 02:25:39.302142 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:25:39.302142 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:25:39.302142 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:25:39.302142 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:25:39.302142 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:25:39.302142 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:25:39.302142 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Aug 08 02:25:39.302142 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Aug 08 02:25:39.303111 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Aug 08 02:25:39.303111 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:25:39.318801 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-562af204-7869-414f-9c33-dd13974c22c4 tempest-VolumesNegativeTest-1884126584 tempest-VolumesNegativeTest-1884126584-project] Returning 400 to user: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' {{(pid=118352) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:25:39.318801 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-562af204-7869-414f-9c33-dd13974c22c4 tempest-VolumesNegativeTest-1884126584 tempest-VolumesNegativeTest-1884126584-project] 10.222.0.27 "POST /compute/v2.1/os-volumes" status: 400 len: 121 microversion: 2.1 time: 0.029045 Aug 08 02:25:39.318801 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 15/26] 10.222.0.27 () {60 vars in 1156 bytes} [Mon Aug 8 02:25:39 2022] POST /compute/v2.1/os-volumes => generated 121 bytes in 30 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Aug 08 02:25:39.327661 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-2fa7688a-bcb5-4460-9de1-e0d0bb36bbd3 tempest-VolumesNegativeTest-1884126584 tempest-VolumesNegativeTest-1884126584-project] Calling method '.version_select at 0x7efc75811f70>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:39.433014 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.wsgi [None req-2fa7688a-bcb5-4460-9de1-e0d0bb36bbd3 tempest-VolumesNegativeTest-1884126584 tempest-VolumesNegativeTest-1884126584-project] HTTP exception thrown: Volume tempest-invalid-794348597 could not be found. Aug 08 02:25:39.439103 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-2fa7688a-bcb5-4460-9de1-e0d0bb36bbd3 tempest-VolumesNegativeTest-1884126584 tempest-VolumesNegativeTest-1884126584-project] Request method failure captured: Aug 08 02:25:39.439103 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: request: DELETE /compute/v2.1/os-volumes/tempest-invalid-794348597 HTTP/1.1 Aug 08 02:25:39.439103 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept: application/json Aug 08 02:25:39.439103 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept-Encoding: identity Aug 08 02:25:39.439103 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Connection: close Aug 08 02:25:39.439103 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Length: 0 Aug 08 02:25:39.439103 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Type: application/json Aug 08 02:25:39.439103 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Host: 10.222.0.27 Aug 08 02:25:39.439103 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Openstack-System-Scope: None Aug 08 02:25:39.439103 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: User-Agent: python-urllib3/1.26.11 Aug 08 02:25:39.439103 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Auth-Token: gAAAAABi8HQgw_1OZEEF-8sA44T9D66ncWMNU0dHE8Alu0u97Euaxv99-3vsK2qgZ4PXcKZnYhK5dIlaxHl8Ki2j-95fV2wYZuK5Sljh_nsGxg855cyWr8cNIF3KfbYbGg1nZijDmm8kxC5jLHotF57JKf1wBvsr5gLFRRD5TxvTvGkspPSPFMw Aug 08 02:25:39.439103 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Id: None Aug 08 02:25:39.439103 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Name: None Aug 08 02:25:39.439103 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Identity-Status: Confirmed Aug 08 02:25:39.439103 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Is-Admin-Project: True Aug 08 02:25:39.439103 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Id: default Aug 08 02:25:39.439103 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Name: Default Aug 08 02:25:39.439103 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Id: d2f4a8558d8242c88877511889b2d6e5 Aug 08 02:25:39.439103 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Name: tempest-VolumesNegativeTest-1884126584 Aug 08 02:25:39.439103 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Role: reader,member Aug 08 02:25:39.439103 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Roles: reader,member Aug 08 02:25:39.440298 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/d2f4a8558d8242c88877511889b2d6e5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/d2f4a8558d8242c88877511889b2d6e5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_d2f4a8558d8242c88877511889b2d6e5", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/d2f4a8558d8242c88877511889b2d6e5"}]}] Aug 08 02:25:39.440298 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant: tempest-VolumesNegativeTest-1884126584 Aug 08 02:25:39.440298 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Id: d2f4a8558d8242c88877511889b2d6e5 Aug 08 02:25:39.440298 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Name: tempest-VolumesNegativeTest-1884126584 Aug 08 02:25:39.440298 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User: tempest-VolumesNegativeTest-1884126584-project Aug 08 02:25:39.440298 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Id: default Aug 08 02:25:39.440298 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Name: Default Aug 08 02:25:39.440298 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Id: 0925fdcf05e442e3924759a24a3ee099 Aug 08 02:25:39.440298 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Name: tempest-VolumesNegativeTest-1884126584-project Aug 08 02:25:39.440298 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: method: Aug 08 02:25:39.440298 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: exception: Volume tempest-invalid-794348597 could not be found. Aug 08 02:25:39.440298 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: action_args: {'id': 'tempest-invalid-794348597'} Aug 08 02:25:39.440298 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:25:39.440995 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:25:39.440995 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 08 02:25:39.440995 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 08 02:25:39.440995 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Aug 08 02:25:39.440995 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Aug 08 02:25:39.440995 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Aug 08 02:25:39.440995 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return self._delete(loc) Aug 08 02:25:39.440995 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Aug 08 02:25:39.440995 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Aug 08 02:25:39.440995 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Aug 08 02:25:39.440995 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Aug 08 02:25:39.440995 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 08 02:25:39.440995 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 08 02:25:39.440995 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 08 02:25:39.440995 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 08 02:25:39.440995 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume tempest-invalid-794348597 could not be found. (HTTP 404) (Request-ID: req-1b765455-18ef-42f4-a9cd-8a9542fd9f17) Aug 08 02:25:39.440995 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:25:39.440995 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:25:39.440995 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:25:39.441966 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:25:39.441966 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Aug 08 02:25:39.441966 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Aug 08 02:25:39.441966 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 08 02:25:39.441966 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 08 02:25:39.441966 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Aug 08 02:25:39.441966 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Aug 08 02:25:39.441966 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 08 02:25:39.441966 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 08 02:25:39.441966 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 08 02:25:39.441966 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 08 02:25:39.441966 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Aug 08 02:25:39.441966 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Aug 08 02:25:39.441966 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Aug 08 02:25:39.441966 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return self._delete(loc) Aug 08 02:25:39.441966 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Aug 08 02:25:39.441966 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Aug 08 02:25:39.441966 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Aug 08 02:25:39.441966 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Aug 08 02:25:39.442971 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 08 02:25:39.442971 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 08 02:25:39.442971 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 08 02:25:39.442971 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 08 02:25:39.442971 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume tempest-invalid-794348597 could not be found. Aug 08 02:25:39.442971 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:25:39.442971 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:25:39.442971 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:25:39.442971 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:25:39.442971 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:25:39.442971 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:25:39.442971 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:25:39.442971 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:25:39.442971 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 08 02:25:39.442971 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 08 02:25:39.442971 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:25:39.442971 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:25:39.442971 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Aug 08 02:25:39.442971 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:25:39.442971 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume tempest-invalid-794348597 could not be found. Aug 08 02:25:39.445642 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:25:39.445642 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:25:39.445642 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:25:39.445642 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:25:39.445642 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:25:39.445642 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:25:39.445642 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:25:39.445642 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:25:39.445642 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume tempest-invalid-794348597 could not be found. Aug 08 02:25:39.445642 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:25:39.445642 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-2fa7688a-bcb5-4460-9de1-e0d0bb36bbd3 tempest-VolumesNegativeTest-1884126584 tempest-VolumesNegativeTest-1884126584-project] Returning 404 to user: Volume tempest-invalid-794348597 could not be found. {{(pid=118352) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:25:39.445642 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-2fa7688a-bcb5-4460-9de1-e0d0bb36bbd3 tempest-VolumesNegativeTest-1884126584 tempest-VolumesNegativeTest-1884126584-project] 10.222.0.27 "DELETE /compute/v2.1/os-volumes/tempest-invalid-794348597" status: 404 len: 98 microversion: 2.1 time: 0.120399 Aug 08 02:25:39.449008 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 16/27] 10.222.0.27 () {58 vars in 1215 bytes} [Mon Aug 8 02:25:39 2022] DELETE /compute/v2.1/os-volumes/tempest-invalid-794348597 => generated 98 bytes in 128 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Aug 08 02:25:39.467519 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-b9bfd8c8-1ec2-4300-a511-494bca82cfa2 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-2026524282", "imageRef": "30833f47-5987-41ca-b5ee-ad5ecafbb265", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "23dedbcc-b8b7-4ca6-a0c0-944d2accdccd"}], "security_groups": [{"name": "tempest-securitygroup--1230300632"}], "key_name": "tempest-keypair-2085665456", "user_data": "}} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:25:39.798677 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.network.neutron [None req-b9bfd8c8-1ec2-4300-a511-494bca82cfa2 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] validate_networks() for [('23dedbcc-b8b7-4ca6-a0c0-944d2accdccd', None, None, None, None, None)] {{(pid=118352) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 08 02:25:40.517661 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.virt.hardware [None req-f3f2d311-7ea3-4c81-abc4-ceedc933c30f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Flavor limits 0:0:0 {{(pid=118351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 08 02:25:40.518197 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.virt.hardware [None req-f3f2d311-7ea3-4c81-abc4-ceedc933c30f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Image limits 0:0:0 {{(pid=118351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 08 02:25:40.518555 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.virt.hardware [None req-f3f2d311-7ea3-4c81-abc4-ceedc933c30f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Flavor pref 0:0:0 {{(pid=118351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 08 02:25:40.518970 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.virt.hardware [None req-f3f2d311-7ea3-4c81-abc4-ceedc933c30f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Image pref 0:0:0 {{(pid=118351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 08 02:25:40.519389 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.virt.hardware [None req-f3f2d311-7ea3-4c81-abc4-ceedc933c30f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 08 02:25:40.549059 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-b9bfd8c8-1ec2-4300-a511-494bca82cfa2 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Flavor limits 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 08 02:25:40.549680 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-b9bfd8c8-1ec2-4300-a511-494bca82cfa2 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Image limits 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 08 02:25:40.549781 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-b9bfd8c8-1ec2-4300-a511-494bca82cfa2 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Flavor pref 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 08 02:25:40.550255 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-b9bfd8c8-1ec2-4300-a511-494bca82cfa2 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Image pref 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 08 02:25:40.550734 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-b9bfd8c8-1ec2-4300-a511-494bca82cfa2 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 08 02:25:40.563572 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b9bfd8c8-1ec2-4300-a511-494bca82cfa2 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:40.566789 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b9bfd8c8-1ec2-4300-a511-494bca82cfa2 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:40.567476 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b9bfd8c8-1ec2-4300-a511-494bca82cfa2 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:40.568456 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b9bfd8c8-1ec2-4300-a511-494bca82cfa2 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:40.569139 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b9bfd8c8-1ec2-4300-a511-494bca82cfa2 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:40.569772 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b9bfd8c8-1ec2-4300-a511-494bca82cfa2 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:40.571582 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f3f2d311-7ea3-4c81-abc4-ceedc933c30f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:40.572739 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f3f2d311-7ea3-4c81-abc4-ceedc933c30f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:40.573578 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f3f2d311-7ea3-4c81-abc4-ceedc933c30f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:40.574484 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f3f2d311-7ea3-4c81-abc4-ceedc933c30f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:40.576600 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f3f2d311-7ea3-4c81-abc4-ceedc933c30f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:40.578973 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f3f2d311-7ea3-4c81-abc4-ceedc933c30f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:40.606540 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.quota [None req-b9bfd8c8-1ec2-4300-a511-494bca82cfa2 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Getting quotas for project ee8baa7d6e294e27ae2fb12eded3709d. Resources: {'ram', 'cores', 'instances'} {{(pid=118352) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 08 02:25:40.612494 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.quota [None req-b9bfd8c8-1ec2-4300-a511-494bca82cfa2 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Getting quotas for user eb4c92edcb024e32a549650eb85f6952 and project ee8baa7d6e294e27ae2fb12eded3709d. Resources: {'ram', 'cores', 'instances'} {{(pid=118352) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 08 02:25:40.626627 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-b9bfd8c8-1ec2-4300-a511-494bca82cfa2 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Going to run 1 instances... {{(pid=118352) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Aug 08 02:25:40.629044 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.quota [None req-f3f2d311-7ea3-4c81-abc4-ceedc933c30f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Getting quotas for project 7caa43bf43ad4e84b4b648e9ab33bd7f. Resources: {'ram', 'cores', 'instances'} {{(pid=118351) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 08 02:25:40.629297 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-b9bfd8c8-1ec2-4300-a511-494bca82cfa2 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='30833f47-5987-41ca-b5ee-ad5ecafbb265',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118352) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Aug 08 02:25:40.645527 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.quota [None req-f3f2d311-7ea3-4c81-abc4-ceedc933c30f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Getting quotas for user d451fb2f26ce4f50b010fb2a387c4bf3 and project 7caa43bf43ad4e84b4b648e9ab33bd7f. Resources: {'ram', 'cores', 'instances'} {{(pid=118351) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 08 02:25:40.658458 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-f3f2d311-7ea3-4c81-abc4-ceedc933c30f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Going to run 1 instances... {{(pid=118351) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Aug 08 02:25:40.660638 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-f3f2d311-7ea3-4c81-abc4-ceedc933c30f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='30833f47-5987-41ca-b5ee-ad5ecafbb265',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118351) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Aug 08 02:25:41.036008 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-b9bfd8c8-1ec2-4300-a511-494bca82cfa2 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 1.581713 Aug 08 02:25:41.043350 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 17/28] 10.222.0.27 () {60 vars in 1147 bytes} [Mon Aug 8 02:25:39 2022] POST /compute/v2.1/servers => generated 402 bytes in 1584 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 08 02:25:41.052754 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-e0632626-8b42-4b9f-a4f8-2e31a065a4c7 tempest-VolumesNegativeTest-1884126584 tempest-VolumesNegativeTest-1884126584-project] 10.222.0.27 "DELETE /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.004596 Aug 08 02:25:41.053286 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-f3f2d311-7ea3-4c81-abc4-ceedc933c30f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.20 time: 2.271352 Aug 08 02:25:41.053889 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 18/29] 10.222.0.27 () {58 vars in 1140 bytes} [Mon Aug 8 02:25:41 2022] DELETE /compute/v2.1/os-volumes/ => generated 112 bytes in 7 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Aug 08 02:25:41.058571 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 12/30] 10.222.0.27 () {62 vars in 1188 bytes} [Mon Aug 8 02:25:38 2022] POST /compute/v2.1/servers => generated 402 bytes in 2286 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Aug 08 02:25:41.066529 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-1d301754-4413-47d1-b033-47ee96dcb628 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:41.067155 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-1d301754-4413-47d1-b033-47ee96dcb628 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:41.068119 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-6bd8f619-adfe-4d07-a9b7-3403d4833214 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:41.069242 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-6bd8f619-adfe-4d07-a9b7-3403d4833214 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: 95ae6b45-35d7-43a7-a021-91bb545cac4e] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:41.079725 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6bd8f619-adfe-4d07-a9b7-3403d4833214 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:41.080434 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6bd8f619-adfe-4d07-a9b7-3403d4833214 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:41.081131 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6bd8f619-adfe-4d07-a9b7-3403d4833214 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:41.082110 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6bd8f619-adfe-4d07-a9b7-3403d4833214 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:41.082968 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6bd8f619-adfe-4d07-a9b7-3403d4833214 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:41.083687 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6bd8f619-adfe-4d07-a9b7-3403d4833214 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:41.128143 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1d301754-4413-47d1-b033-47ee96dcb628 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:41.128143 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1d301754-4413-47d1-b033-47ee96dcb628 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:41.128143 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1d301754-4413-47d1-b033-47ee96dcb628 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:41.128143 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1d301754-4413-47d1-b033-47ee96dcb628 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:41.130499 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1d301754-4413-47d1-b033-47ee96dcb628 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:41.130499 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1d301754-4413-47d1-b033-47ee96dcb628 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:41.252292 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-1d301754-4413-47d1-b033-47ee96dcb628 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:41.267391 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-1d301754-4413-47d1-b033-47ee96dcb628 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1280 microversion: 2.1 time: 0.212377 Aug 08 02:25:41.272724 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 13/31] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:41 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1280 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:41.288388 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-42ac564b-a90d-4e5c-8500-2605144ff736 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:41.289119 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-42ac564b-a90d-4e5c-8500-2605144ff736 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] [instance: 0fc3bfe1-b902-4a65-8c41-81e6f49112ae] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:41.307961 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-42ac564b-a90d-4e5c-8500-2605144ff736 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:41.307961 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-42ac564b-a90d-4e5c-8500-2605144ff736 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:41.307961 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-42ac564b-a90d-4e5c-8500-2605144ff736 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:41.308176 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-42ac564b-a90d-4e5c-8500-2605144ff736 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:41.311538 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-42ac564b-a90d-4e5c-8500-2605144ff736 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:41.313192 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-42ac564b-a90d-4e5c-8500-2605144ff736 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:41.406109 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-6bd8f619-adfe-4d07-a9b7-3403d4833214 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:41.468097 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-6bd8f619-adfe-4d07-a9b7-3403d4833214 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e" status: 200 len: 1245 microversion: 2.1 time: 0.403413 Aug 08 02:25:41.475698 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 19/32] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:41 2022] GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e => generated 1245 bytes in 415 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:41.495641 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-cc88877b-2880-4f5a-a4bb-235a5b35e729 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:41.508074 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-cc88877b-2880-4f5a-a4bb-235a5b35e729 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:41.539351 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-42ac564b-a90d-4e5c-8500-2605144ff736 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2aeb52d51d3f44319bb4e114858222f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5515114a058944a49182063213de54d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:41.569206 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-42ac564b-a90d-4e5c-8500-2605144ff736 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] 10.222.0.27 "GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae" status: 200 len: 1218 microversion: 2.1 time: 0.286946 Aug 08 02:25:41.569206 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 14/33] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:41 2022] GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae => generated 1218 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:41.569206 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-c00bb612-0391-4bf1-b759-143eaedab53a tempest-VolumesNegativeTest-1884126584 tempest-VolumesNegativeTest-1884126584-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.005807 Aug 08 02:25:41.569206 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 15/34] 10.222.0.27 () {58 vars in 1137 bytes} [Mon Aug 8 02:25:41 2022] GET /compute/v2.1/os-volumes/ => generated 112 bytes in 8 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Aug 08 02:25:41.571982 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-de387626-1b51-4c01-94f2-59b752de8034 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:41.572986 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-de387626-1b51-4c01-94f2-59b752de8034 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:41.680491 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-cc88877b-2880-4f5a-a4bb-235a5b35e729 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:41.704546 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-cc88877b-2880-4f5a-a4bb-235a5b35e729 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1238 microversion: 2.1 time: 0.227587 Aug 08 02:25:41.709787 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 20/35] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:41 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1238 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:41.715270 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-6081e5c9-fc59-4dee-95b6-8087b45a4071 tempest-VolumesNegativeTest-1884126584 tempest-VolumesNegativeTest-1884126584-project] Calling method '.version_select at 0x7efc74579820>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:41.743084 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-de387626-1b51-4c01-94f2-59b752de8034 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:41.757021 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-de387626-1b51-4c01-94f2-59b752de8034 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:41.762632 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-de387626-1b51-4c01-94f2-59b752de8034 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:41.764738 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-de387626-1b51-4c01-94f2-59b752de8034 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1282 microversion: 2.20 time: 0.197801 Aug 08 02:25:41.771869 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 16/36] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:25:41 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1282 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:25:42.294436 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-36cef3c9-f893-4db7-86ce-2c524cf30e75 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:42.295076 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-36cef3c9-f893-4db7-86ce-2c524cf30e75 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:42.308838 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-36cef3c9-f893-4db7-86ce-2c524cf30e75 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:42.309343 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-36cef3c9-f893-4db7-86ce-2c524cf30e75 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:42.310779 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-36cef3c9-f893-4db7-86ce-2c524cf30e75 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:42.311545 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-36cef3c9-f893-4db7-86ce-2c524cf30e75 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:42.311988 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-36cef3c9-f893-4db7-86ce-2c524cf30e75 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:42.312443 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-36cef3c9-f893-4db7-86ce-2c524cf30e75 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:42.640870 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-36cef3c9-f893-4db7-86ce-2c524cf30e75 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:42.675696 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-36cef3c9-f893-4db7-86ce-2c524cf30e75 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1322 microversion: 2.1 time: 0.387070 Aug 08 02:25:42.679651 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 17/37] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:42 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1322 bytes in 393 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:25:42.705148 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-407e973a-9287-436c-9649-493863a5d30f tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:42.715154 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-407e973a-9287-436c-9649-493863a5d30f tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: 95ae6b45-35d7-43a7-a021-91bb545cac4e] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:42.722214 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-407e973a-9287-436c-9649-493863a5d30f tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:42.723100 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-407e973a-9287-436c-9649-493863a5d30f tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:42.723829 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-407e973a-9287-436c-9649-493863a5d30f tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:42.725082 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-407e973a-9287-436c-9649-493863a5d30f tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:42.727421 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-407e973a-9287-436c-9649-493863a5d30f tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:42.728039 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-407e973a-9287-436c-9649-493863a5d30f tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:42.894438 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.wsgi [None req-6081e5c9-fc59-4dee-95b6-8087b45a4071 tempest-VolumesNegativeTest-1884126584 tempest-VolumesNegativeTest-1884126584-project] HTTP exception thrown: Volume 63746aa5-63e8-465e-b937-261840bddea9 could not be found. Aug 08 02:25:42.909282 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-6081e5c9-fc59-4dee-95b6-8087b45a4071 tempest-VolumesNegativeTest-1884126584 tempest-VolumesNegativeTest-1884126584-project] Request method failure captured: Aug 08 02:25:42.909282 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: request: DELETE /compute/v2.1/os-volumes/63746aa5-63e8-465e-b937-261840bddea9 HTTP/1.1 Aug 08 02:25:42.909282 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept: application/json Aug 08 02:25:42.909282 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept-Encoding: identity Aug 08 02:25:42.909282 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Connection: close Aug 08 02:25:42.909282 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Length: 0 Aug 08 02:25:42.909282 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Type: application/json Aug 08 02:25:42.909282 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Host: 10.222.0.27 Aug 08 02:25:42.909282 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Openstack-System-Scope: None Aug 08 02:25:42.909282 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: User-Agent: python-urllib3/1.26.11 Aug 08 02:25:42.909282 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Auth-Token: gAAAAABi8HQgw_1OZEEF-8sA44T9D66ncWMNU0dHE8Alu0u97Euaxv99-3vsK2qgZ4PXcKZnYhK5dIlaxHl8Ki2j-95fV2wYZuK5Sljh_nsGxg855cyWr8cNIF3KfbYbGg1nZijDmm8kxC5jLHotF57JKf1wBvsr5gLFRRD5TxvTvGkspPSPFMw Aug 08 02:25:42.909282 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Id: None Aug 08 02:25:42.909282 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Name: None Aug 08 02:25:42.909282 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Identity-Status: Confirmed Aug 08 02:25:42.909282 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Is-Admin-Project: True Aug 08 02:25:42.909282 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Id: default Aug 08 02:25:42.909282 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Name: Default Aug 08 02:25:42.909282 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Id: d2f4a8558d8242c88877511889b2d6e5 Aug 08 02:25:42.909282 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Name: tempest-VolumesNegativeTest-1884126584 Aug 08 02:25:42.909282 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Role: reader,member Aug 08 02:25:42.909282 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Roles: reader,member Aug 08 02:25:42.911052 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/d2f4a8558d8242c88877511889b2d6e5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/d2f4a8558d8242c88877511889b2d6e5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_d2f4a8558d8242c88877511889b2d6e5", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/d2f4a8558d8242c88877511889b2d6e5"}]}] Aug 08 02:25:42.911052 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant: tempest-VolumesNegativeTest-1884126584 Aug 08 02:25:42.911052 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Id: d2f4a8558d8242c88877511889b2d6e5 Aug 08 02:25:42.911052 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Name: tempest-VolumesNegativeTest-1884126584 Aug 08 02:25:42.911052 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User: tempest-VolumesNegativeTest-1884126584-project Aug 08 02:25:42.911052 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Id: default Aug 08 02:25:42.911052 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Name: Default Aug 08 02:25:42.911052 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Id: 0925fdcf05e442e3924759a24a3ee099 Aug 08 02:25:42.911052 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Name: tempest-VolumesNegativeTest-1884126584-project Aug 08 02:25:42.911052 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: method: Aug 08 02:25:42.911052 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: exception: Volume 63746aa5-63e8-465e-b937-261840bddea9 could not be found. Aug 08 02:25:42.911052 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: action_args: {'id': '63746aa5-63e8-465e-b937-261840bddea9'} Aug 08 02:25:42.911052 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:25:42.911963 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:25:42.911963 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 08 02:25:42.911963 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 08 02:25:42.911963 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Aug 08 02:25:42.911963 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Aug 08 02:25:42.911963 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Aug 08 02:25:42.911963 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return self._delete(loc) Aug 08 02:25:42.911963 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Aug 08 02:25:42.911963 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Aug 08 02:25:42.911963 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Aug 08 02:25:42.911963 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Aug 08 02:25:42.911963 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 08 02:25:42.911963 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 08 02:25:42.911963 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 08 02:25:42.911963 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 08 02:25:42.911963 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 63746aa5-63e8-465e-b937-261840bddea9 could not be found. (HTTP 404) (Request-ID: req-30ad9669-d854-4cfe-92e6-cd6c58bbbda0) Aug 08 02:25:42.911963 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:25:42.911963 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:25:42.911963 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:25:42.914106 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:25:42.914106 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Aug 08 02:25:42.914106 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Aug 08 02:25:42.914106 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 08 02:25:42.914106 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 08 02:25:42.914106 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Aug 08 02:25:42.914106 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Aug 08 02:25:42.914106 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 08 02:25:42.914106 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 08 02:25:42.914106 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 08 02:25:42.914106 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 08 02:25:42.914106 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Aug 08 02:25:42.914106 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Aug 08 02:25:42.914106 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Aug 08 02:25:42.914106 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return self._delete(loc) Aug 08 02:25:42.914106 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Aug 08 02:25:42.914106 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Aug 08 02:25:42.914106 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Aug 08 02:25:42.914106 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Aug 08 02:25:42.915454 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 08 02:25:42.915454 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 08 02:25:42.915454 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 08 02:25:42.915454 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 08 02:25:42.915454 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 63746aa5-63e8-465e-b937-261840bddea9 could not be found. Aug 08 02:25:42.915454 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:25:42.915454 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:25:42.915454 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:25:42.915454 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:25:42.915454 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:25:42.915454 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:25:42.915454 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:25:42.915454 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:25:42.915454 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 08 02:25:42.915454 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 08 02:25:42.915454 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:25:42.915454 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:25:42.915454 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Aug 08 02:25:42.915454 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:25:42.916458 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 63746aa5-63e8-465e-b937-261840bddea9 could not be found. Aug 08 02:25:42.916458 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:25:42.916458 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:25:42.916458 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:25:42.916458 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:25:42.916458 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:25:42.916458 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:25:42.916458 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:25:42.916458 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:25:42.916458 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 63746aa5-63e8-465e-b937-261840bddea9 could not be found. Aug 08 02:25:42.916458 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:25:42.929232 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-6081e5c9-fc59-4dee-95b6-8087b45a4071 tempest-VolumesNegativeTest-1884126584 tempest-VolumesNegativeTest-1884126584-project] Returning 404 to user: Volume 63746aa5-63e8-465e-b937-261840bddea9 could not be found. {{(pid=118352) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:25:42.932393 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-6081e5c9-fc59-4dee-95b6-8087b45a4071 tempest-VolumesNegativeTest-1884126584 tempest-VolumesNegativeTest-1884126584-project] 10.222.0.27 "DELETE /compute/v2.1/os-volumes/63746aa5-63e8-465e-b937-261840bddea9" status: 404 len: 109 microversion: 2.1 time: 1.220419 Aug 08 02:25:42.933202 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 21/38] 10.222.0.27 () {58 vars in 1248 bytes} [Mon Aug 8 02:25:41 2022] DELETE /compute/v2.1/os-volumes/63746aa5-63e8-465e-b937-261840bddea9 => generated 109 bytes in 1223 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 08 02:25:42.962980 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-407e973a-9287-436c-9649-493863a5d30f tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:42.977029 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-ae6d60f8-168e-40b0-b3fd-cab66dcb66b4 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:42.977863 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-ae6d60f8-168e-40b0-b3fd-cab66dcb66b4 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] [instance: 0fc3bfe1-b902-4a65-8c41-81e6f49112ae] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:42.988051 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-407e973a-9287-436c-9649-493863a5d30f tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e" status: 200 len: 1293 microversion: 2.1 time: 0.305722 Aug 08 02:25:42.994265 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 18/39] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:42 2022] GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e => generated 1293 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:43.004637 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-f7fa0c88-c0f9-4c03-b72e-5682c9398a7a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:43.014731 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ae6d60f8-168e-40b0-b3fd-cab66dcb66b4 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:43.015249 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ae6d60f8-168e-40b0-b3fd-cab66dcb66b4 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:43.015704 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ae6d60f8-168e-40b0-b3fd-cab66dcb66b4 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:43.016430 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ae6d60f8-168e-40b0-b3fd-cab66dcb66b4 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:43.026880 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-f7fa0c88-c0f9-4c03-b72e-5682c9398a7a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:43.031529 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ae6d60f8-168e-40b0-b3fd-cab66dcb66b4 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:43.031529 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ae6d60f8-168e-40b0-b3fd-cab66dcb66b4 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.014s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:43.038871 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f7fa0c88-c0f9-4c03-b72e-5682c9398a7a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:43.039371 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f7fa0c88-c0f9-4c03-b72e-5682c9398a7a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:43.039810 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f7fa0c88-c0f9-4c03-b72e-5682c9398a7a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:43.040526 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f7fa0c88-c0f9-4c03-b72e-5682c9398a7a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:43.041017 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f7fa0c88-c0f9-4c03-b72e-5682c9398a7a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:43.041447 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f7fa0c88-c0f9-4c03-b72e-5682c9398a7a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:43.201879 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-ae6d60f8-168e-40b0-b3fd-cab66dcb66b4 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2aeb52d51d3f44319bb4e114858222f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5515114a058944a49182063213de54d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:43.212607 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-ae6d60f8-168e-40b0-b3fd-cab66dcb66b4 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] 10.222.0.27 "GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae" status: 200 len: 1266 microversion: 2.1 time: 0.273775 Aug 08 02:25:43.222134 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 22/40] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:42 2022] GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae => generated 1266 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:43.232874 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-dd59dfe6-dd87-4245-8a31-b0894a9085c0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:43.237760 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-dd59dfe6-dd87-4245-8a31-b0894a9085c0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:43.255021 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dd59dfe6-dd87-4245-8a31-b0894a9085c0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:43.255849 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dd59dfe6-dd87-4245-8a31-b0894a9085c0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:43.255977 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-f7fa0c88-c0f9-4c03-b72e-5682c9398a7a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:43.256086 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dd59dfe6-dd87-4245-8a31-b0894a9085c0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:43.256900 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dd59dfe6-dd87-4245-8a31-b0894a9085c0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:43.257414 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dd59dfe6-dd87-4245-8a31-b0894a9085c0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:43.257880 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dd59dfe6-dd87-4245-8a31-b0894a9085c0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:43.274716 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-f7fa0c88-c0f9-4c03-b72e-5682c9398a7a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1242 microversion: 2.1 time: 0.273552 Aug 08 02:25:43.277861 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 19/41] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:42 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1242 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:43.289481 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-ded9dc9a-3b83-43a4-89c0-43a289eafed5 tempest-VolumesNegativeTest-1884126584 tempest-VolumesNegativeTest-1884126584-project] Calling method '.version_select at 0x7efc74657820>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:43.357445 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.wsgi [None req-ded9dc9a-3b83-43a4-89c0-43a289eafed5 tempest-VolumesNegativeTest-1884126584 tempest-VolumesNegativeTest-1884126584-project] HTTP exception thrown: Volume dc458cb5-1945-4c8f-8b5e-3219c7e8a979 could not be found. Aug 08 02:25:43.362254 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-ded9dc9a-3b83-43a4-89c0-43a289eafed5 tempest-VolumesNegativeTest-1884126584 tempest-VolumesNegativeTest-1884126584-project] Request method failure captured: Aug 08 02:25:43.362254 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: request: GET /compute/v2.1/os-volumes/dc458cb5-1945-4c8f-8b5e-3219c7e8a979 HTTP/1.1 Aug 08 02:25:43.362254 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Accept: application/json Aug 08 02:25:43.362254 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Accept-Encoding: identity Aug 08 02:25:43.362254 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Connection: close Aug 08 02:25:43.362254 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Content-Length: 0 Aug 08 02:25:43.362254 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Content-Type: application/json Aug 08 02:25:43.362254 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Host: 10.222.0.27 Aug 08 02:25:43.362254 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Openstack-System-Scope: None Aug 08 02:25:43.362254 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: User-Agent: python-urllib3/1.26.11 Aug 08 02:25:43.362254 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Auth-Token: gAAAAABi8HQgw_1OZEEF-8sA44T9D66ncWMNU0dHE8Alu0u97Euaxv99-3vsK2qgZ4PXcKZnYhK5dIlaxHl8Ki2j-95fV2wYZuK5Sljh_nsGxg855cyWr8cNIF3KfbYbGg1nZijDmm8kxC5jLHotF57JKf1wBvsr5gLFRRD5TxvTvGkspPSPFMw Aug 08 02:25:43.362254 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Domain-Id: None Aug 08 02:25:43.362254 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Domain-Name: None Aug 08 02:25:43.362254 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Identity-Status: Confirmed Aug 08 02:25:43.362254 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Is-Admin-Project: True Aug 08 02:25:43.362254 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Domain-Id: default Aug 08 02:25:43.362254 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Domain-Name: Default Aug 08 02:25:43.362254 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Id: d2f4a8558d8242c88877511889b2d6e5 Aug 08 02:25:43.362254 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Name: tempest-VolumesNegativeTest-1884126584 Aug 08 02:25:43.362254 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Role: reader,member Aug 08 02:25:43.362254 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Roles: reader,member Aug 08 02:25:43.363425 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/d2f4a8558d8242c88877511889b2d6e5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/d2f4a8558d8242c88877511889b2d6e5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_d2f4a8558d8242c88877511889b2d6e5", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/d2f4a8558d8242c88877511889b2d6e5"}]}] Aug 08 02:25:43.363425 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant: tempest-VolumesNegativeTest-1884126584 Aug 08 02:25:43.363425 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant-Id: d2f4a8558d8242c88877511889b2d6e5 Aug 08 02:25:43.363425 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant-Name: tempest-VolumesNegativeTest-1884126584 Aug 08 02:25:43.363425 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User: tempest-VolumesNegativeTest-1884126584-project Aug 08 02:25:43.363425 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Domain-Id: default Aug 08 02:25:43.363425 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Domain-Name: Default Aug 08 02:25:43.363425 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Id: 0925fdcf05e442e3924759a24a3ee099 Aug 08 02:25:43.363425 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Name: tempest-VolumesNegativeTest-1884126584-project Aug 08 02:25:43.363425 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: method: Aug 08 02:25:43.363425 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: exception: Volume dc458cb5-1945-4c8f-8b5e-3219c7e8a979 could not be found. Aug 08 02:25:43.363425 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: action_args: {'id': 'dc458cb5-1945-4c8f-8b5e-3219c7e8a979'} Aug 08 02:25:43.363425 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:25:43.364172 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:25:43.364172 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 08 02:25:43.364172 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 08 02:25:43.364172 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Aug 08 02:25:43.364172 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi item = cinderclient( Aug 08 02:25:43.364172 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Aug 08 02:25:43.364172 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Aug 08 02:25:43.364172 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 08 02:25:43.364172 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 08 02:25:43.364172 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 08 02:25:43.364172 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 08 02:25:43.364172 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 08 02:25:43.364172 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 08 02:25:43.364172 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 08 02:25:43.364172 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 08 02:25:43.364172 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume dc458cb5-1945-4c8f-8b5e-3219c7e8a979 could not be found. (HTTP 404) (Request-ID: req-38a52cb6-a3a4-4482-8127-942400e73011) Aug 08 02:25:43.364172 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:25:43.364172 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:25:43.364172 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:25:43.365331 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:25:43.365331 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Aug 08 02:25:43.365331 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Aug 08 02:25:43.365331 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 08 02:25:43.365331 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 08 02:25:43.365331 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Aug 08 02:25:43.365331 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Aug 08 02:25:43.365331 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 08 02:25:43.365331 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 08 02:25:43.365331 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 08 02:25:43.365331 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 08 02:25:43.365331 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Aug 08 02:25:43.365331 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi item = cinderclient( Aug 08 02:25:43.365331 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Aug 08 02:25:43.365331 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Aug 08 02:25:43.365331 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 08 02:25:43.365331 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 08 02:25:43.365331 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 08 02:25:43.365331 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 08 02:25:43.366338 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 08 02:25:43.366338 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 08 02:25:43.366338 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 08 02:25:43.366338 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 08 02:25:43.366338 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume dc458cb5-1945-4c8f-8b5e-3219c7e8a979 could not be found. Aug 08 02:25:43.366338 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:25:43.366338 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:25:43.366338 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:25:43.366338 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:25:43.366338 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:25:43.366338 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:25:43.366338 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:25:43.366338 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:25:43.366338 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 08 02:25:43.366338 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 08 02:25:43.366338 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:25:43.366338 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:25:43.366338 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Aug 08 02:25:43.366338 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:25:43.367428 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume dc458cb5-1945-4c8f-8b5e-3219c7e8a979 could not be found. Aug 08 02:25:43.367428 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:25:43.367428 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:25:43.367428 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:25:43.367428 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:25:43.367428 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:25:43.367428 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:25:43.367428 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:25:43.367428 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:25:43.367428 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume dc458cb5-1945-4c8f-8b5e-3219c7e8a979 could not be found. Aug 08 02:25:43.367428 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:25:43.367428 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-ded9dc9a-3b83-43a4-89c0-43a289eafed5 tempest-VolumesNegativeTest-1884126584 tempest-VolumesNegativeTest-1884126584-project] Returning 404 to user: Volume dc458cb5-1945-4c8f-8b5e-3219c7e8a979 could not be found. {{(pid=118351) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:25:43.367428 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-ded9dc9a-3b83-43a4-89c0-43a289eafed5 tempest-VolumesNegativeTest-1884126584 tempest-VolumesNegativeTest-1884126584-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/dc458cb5-1945-4c8f-8b5e-3219c7e8a979" status: 404 len: 109 microversion: 2.1 time: 0.084267 Aug 08 02:25:43.368214 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 20/42] 10.222.0.27 () {58 vars in 1245 bytes} [Mon Aug 8 02:25:43 2022] GET /compute/v2.1/os-volumes/dc458cb5-1945-4c8f-8b5e-3219c7e8a979 => generated 109 bytes in 89 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 08 02:25:43.409629 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-dd59dfe6-dd87-4245-8a31-b0894a9085c0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:43.434718 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-dd59dfe6-dd87-4245-8a31-b0894a9085c0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:43.439011 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-dd59dfe6-dd87-4245-8a31-b0894a9085c0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:43.443290 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-dd59dfe6-dd87-4245-8a31-b0894a9085c0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1286 microversion: 2.20 time: 0.212337 Aug 08 02:25:43.469040 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 23/43] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:25:43 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1286 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:25:43.710394 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-e1a92ae6-e126-41a7-8421-0eb129f14a53 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:43.712180 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-e1a92ae6-e126-41a7-8421-0eb129f14a53 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:43.721818 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e1a92ae6-e126-41a7-8421-0eb129f14a53 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:43.722363 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e1a92ae6-e126-41a7-8421-0eb129f14a53 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:43.722835 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e1a92ae6-e126-41a7-8421-0eb129f14a53 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:43.723588 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e1a92ae6-e126-41a7-8421-0eb129f14a53 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:43.724051 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e1a92ae6-e126-41a7-8421-0eb129f14a53 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:43.724681 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e1a92ae6-e126-41a7-8421-0eb129f14a53 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:44.017822 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-5d665615-c3b2-4797-acb5-dfd89623c80c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:44.019244 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-5d665615-c3b2-4797-acb5-dfd89623c80c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: 95ae6b45-35d7-43a7-a021-91bb545cac4e] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:44.050190 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5d665615-c3b2-4797-acb5-dfd89623c80c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:44.064268 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5d665615-c3b2-4797-acb5-dfd89623c80c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:44.064268 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5d665615-c3b2-4797-acb5-dfd89623c80c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:44.064268 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5d665615-c3b2-4797-acb5-dfd89623c80c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:44.064268 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5d665615-c3b2-4797-acb5-dfd89623c80c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:44.064816 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5d665615-c3b2-4797-acb5-dfd89623c80c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:44.421270 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-e1a92ae6-e126-41a7-8421-0eb129f14a53 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:44.440164 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-e1a92ae6-e126-41a7-8421-0eb129f14a53 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1322 microversion: 2.1 time: 0.739873 Aug 08 02:25:44.444581 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 21/44] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:43 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1322 bytes in 745 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:44.472164 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-2321413f-fcdb-477d-a29c-4ac86b558ab1 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:44.472917 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-2321413f-fcdb-477d-a29c-4ac86b558ab1 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] [instance: 0fc3bfe1-b902-4a65-8c41-81e6f49112ae] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:44.501607 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2321413f-fcdb-477d-a29c-4ac86b558ab1 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:44.507552 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2321413f-fcdb-477d-a29c-4ac86b558ab1 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:44.507552 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2321413f-fcdb-477d-a29c-4ac86b558ab1 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:44.507552 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2321413f-fcdb-477d-a29c-4ac86b558ab1 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:44.507986 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2321413f-fcdb-477d-a29c-4ac86b558ab1 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:44.513195 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2321413f-fcdb-477d-a29c-4ac86b558ab1 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:44.573426 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-5d665615-c3b2-4797-acb5-dfd89623c80c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:44.623166 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-5d665615-c3b2-4797-acb5-dfd89623c80c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e" status: 200 len: 1311 microversion: 2.1 time: 0.607407 Aug 08 02:25:44.629815 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 24/45] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:44 2022] GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e => generated 1311 bytes in 617 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:44.650760 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-4eb0e810-2024-4b4d-b181-3d7b93b5f34d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:44.651499 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-4eb0e810-2024-4b4d-b181-3d7b93b5f34d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:44.682533 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4eb0e810-2024-4b4d-b181-3d7b93b5f34d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:44.684439 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4eb0e810-2024-4b4d-b181-3d7b93b5f34d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:44.688575 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4eb0e810-2024-4b4d-b181-3d7b93b5f34d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:44.689622 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4eb0e810-2024-4b4d-b181-3d7b93b5f34d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:44.690305 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4eb0e810-2024-4b4d-b181-3d7b93b5f34d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:44.691302 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4eb0e810-2024-4b4d-b181-3d7b93b5f34d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:44.844567 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-2321413f-fcdb-477d-a29c-4ac86b558ab1 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2aeb52d51d3f44319bb4e114858222f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5515114a058944a49182063213de54d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:44.879062 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-2321413f-fcdb-477d-a29c-4ac86b558ab1 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] 10.222.0.27 "GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae" status: 200 len: 1274 microversion: 2.1 time: 0.419038 Aug 08 02:25:44.881501 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 22/46] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:44 2022] GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae => generated 1274 bytes in 436 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:44.898485 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-a333aa59-3c66-4b23-9fb5-c47834196b19 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:44.903379 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-a333aa59-3c66-4b23-9fb5-c47834196b19 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:44.926726 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a333aa59-3c66-4b23-9fb5-c47834196b19 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:44.927860 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a333aa59-3c66-4b23-9fb5-c47834196b19 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:44.929847 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a333aa59-3c66-4b23-9fb5-c47834196b19 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:44.932729 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a333aa59-3c66-4b23-9fb5-c47834196b19 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:44.933518 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a333aa59-3c66-4b23-9fb5-c47834196b19 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:44.937316 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a333aa59-3c66-4b23-9fb5-c47834196b19 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:45.062384 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-4eb0e810-2024-4b4d-b181-3d7b93b5f34d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:45.080859 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-4eb0e810-2024-4b4d-b181-3d7b93b5f34d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1290 microversion: 2.1 time: 0.449589 Aug 08 02:25:45.081488 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 25/47] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:44 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1290 bytes in 451 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:45.396321 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-a333aa59-3c66-4b23-9fb5-c47834196b19 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:45.421359 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-a333aa59-3c66-4b23-9fb5-c47834196b19 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:45.423428 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-a333aa59-3c66-4b23-9fb5-c47834196b19 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:45.427036 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-a333aa59-3c66-4b23-9fb5-c47834196b19 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1334 microversion: 2.20 time: 0.544194 Aug 08 02:25:45.428304 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 23/48] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:25:44 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1334 bytes in 546 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:25:45.470684 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-5585e856-488e-454b-b266-7ee97993b322 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:45.479235 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-5585e856-488e-454b-b266-7ee97993b322 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:45.506230 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5585e856-488e-454b-b266-7ee97993b322 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:45.506230 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5585e856-488e-454b-b266-7ee97993b322 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:45.506230 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5585e856-488e-454b-b266-7ee97993b322 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:45.506230 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5585e856-488e-454b-b266-7ee97993b322 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:45.506690 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5585e856-488e-454b-b266-7ee97993b322 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:45.506690 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5585e856-488e-454b-b266-7ee97993b322 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:45.669497 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-a9a8ac0d-91a9-46b5-bd85-f38f02a62bbd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:45.669497 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-a9a8ac0d-91a9-46b5-bd85-f38f02a62bbd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: 95ae6b45-35d7-43a7-a021-91bb545cac4e] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:45.718523 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a9a8ac0d-91a9-46b5-bd85-f38f02a62bbd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:45.722758 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a9a8ac0d-91a9-46b5-bd85-f38f02a62bbd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:45.723309 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a9a8ac0d-91a9-46b5-bd85-f38f02a62bbd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:45.736372 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a9a8ac0d-91a9-46b5-bd85-f38f02a62bbd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:45.737413 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a9a8ac0d-91a9-46b5-bd85-f38f02a62bbd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:45.742543 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a9a8ac0d-91a9-46b5-bd85-f38f02a62bbd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:46.093912 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-5585e856-488e-454b-b266-7ee97993b322 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:46.096305 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-a9a8ac0d-91a9-46b5-bd85-f38f02a62bbd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:46.108470 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-5585e856-488e-454b-b266-7ee97993b322 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1322 microversion: 2.1 time: 0.648415 Aug 08 02:25:46.109522 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 26/49] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:45 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1322 bytes in 650 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:46.125915 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-a6dba9a3-bc22-4207-aa84-cd33a61e0618 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:46.126781 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-a6dba9a3-bc22-4207-aa84-cd33a61e0618 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] [instance: 0fc3bfe1-b902-4a65-8c41-81e6f49112ae] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:46.130267 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_api:SELECT=91,nova_api:INSERT=9 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:46.141156 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a6dba9a3-bc22-4207-aa84-cd33a61e0618 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:46.141919 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a6dba9a3-bc22-4207-aa84-cd33a61e0618 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:46.142528 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a6dba9a3-bc22-4207-aa84-cd33a61e0618 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:46.143398 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a6dba9a3-bc22-4207-aa84-cd33a61e0618 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:46.144467 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-a9a8ac0d-91a9-46b5-bd85-f38f02a62bbd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e" status: 200 len: 1299 microversion: 2.1 time: 0.493863 Aug 08 02:25:46.145281 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 24/50] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:45 2022] GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e => generated 1299 bytes in 496 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:46.158010 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a6dba9a3-bc22-4207-aa84-cd33a61e0618 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:46.163564 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a6dba9a3-bc22-4207-aa84-cd33a61e0618 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:46.163693 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-b2a4874b-d745-492e-a320-feca6629acf7 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:46.163693 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-b2a4874b-d745-492e-a320-feca6629acf7 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:46.186764 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b2a4874b-d745-492e-a320-feca6629acf7 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:46.187534 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b2a4874b-d745-492e-a320-feca6629acf7 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:46.188147 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b2a4874b-d745-492e-a320-feca6629acf7 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:46.192466 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b2a4874b-d745-492e-a320-feca6629acf7 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:46.194214 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b2a4874b-d745-492e-a320-feca6629acf7 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:46.197989 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b2a4874b-d745-492e-a320-feca6629acf7 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:46.492967 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-a6dba9a3-bc22-4207-aa84-cd33a61e0618 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2aeb52d51d3f44319bb4e114858222f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5515114a058944a49182063213de54d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:46.511598 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-a6dba9a3-bc22-4207-aa84-cd33a61e0618 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] 10.222.0.27 "GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae" status: 200 len: 1272 microversion: 2.1 time: 0.400496 Aug 08 02:25:46.515096 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 27/51] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:46 2022] GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae => generated 1272 bytes in 405 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:46.522257 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-17b05995-8c33-481c-b82a-8dce774faf8f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:46.523507 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-17b05995-8c33-481c-b82a-8dce774faf8f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:46.534325 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-b2a4874b-d745-492e-a320-feca6629acf7 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:46.536837 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-17b05995-8c33-481c-b82a-8dce774faf8f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:46.537744 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-17b05995-8c33-481c-b82a-8dce774faf8f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:46.538588 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-17b05995-8c33-481c-b82a-8dce774faf8f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:46.541106 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-17b05995-8c33-481c-b82a-8dce774faf8f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:46.555378 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-17b05995-8c33-481c-b82a-8dce774faf8f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:46.555480 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-b2a4874b-d745-492e-a320-feca6629acf7 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1298 microversion: 2.1 time: 0.403987 Aug 08 02:25:46.555562 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-17b05995-8c33-481c-b82a-8dce774faf8f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:46.568692 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 25/52] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:46 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1298 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:46.824380 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-17b05995-8c33-481c-b82a-8dce774faf8f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:46.888360 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-17b05995-8c33-481c-b82a-8dce774faf8f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:46.901786 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-17b05995-8c33-481c-b82a-8dce774faf8f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:46.903018 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-17b05995-8c33-481c-b82a-8dce774faf8f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1352 microversion: 2.20 time: 0.386181 Aug 08 02:25:46.903747 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 28/53] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:25:46 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1352 bytes in 388 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:25:47.130639 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-59207e78-e810-4a2f-9a05-16986d662556 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:47.131866 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-59207e78-e810-4a2f-9a05-16986d662556 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:47.143052 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_api:SELECT=88,nova_api:INSERT=12 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:47.161543 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-59207e78-e810-4a2f-9a05-16986d662556 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:47.162044 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-59207e78-e810-4a2f-9a05-16986d662556 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:47.163828 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-59207e78-e810-4a2f-9a05-16986d662556 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:47.165479 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-59207e78-e810-4a2f-9a05-16986d662556 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:47.172843 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-f6bf365d-88fa-4027-95eb-65ebc9b8615d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:47.174113 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-f6bf365d-88fa-4027-95eb-65ebc9b8615d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: 95ae6b45-35d7-43a7-a021-91bb545cac4e] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:47.181196 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-59207e78-e810-4a2f-9a05-16986d662556 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:47.183024 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-59207e78-e810-4a2f-9a05-16986d662556 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.016s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:47.190032 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f6bf365d-88fa-4027-95eb-65ebc9b8615d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:47.193319 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f6bf365d-88fa-4027-95eb-65ebc9b8615d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:47.202875 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f6bf365d-88fa-4027-95eb-65ebc9b8615d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:47.207081 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f6bf365d-88fa-4027-95eb-65ebc9b8615d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:47.208770 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f6bf365d-88fa-4027-95eb-65ebc9b8615d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:47.209220 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f6bf365d-88fa-4027-95eb-65ebc9b8615d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:47.484315 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-f6bf365d-88fa-4027-95eb-65ebc9b8615d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:47.506150 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-f6bf365d-88fa-4027-95eb-65ebc9b8615d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e" status: 200 len: 1299 microversion: 2.1 time: 0.338371 Aug 08 02:25:47.529430 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 29/54] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:47 2022] GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e => generated 1299 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:25:47.550571 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-a9bad65a-2044-43fa-843f-cc4e04ad538a tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:47.553883 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-a9bad65a-2044-43fa-843f-cc4e04ad538a tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] [instance: 0fc3bfe1-b902-4a65-8c41-81e6f49112ae] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:47.573983 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a9bad65a-2044-43fa-843f-cc4e04ad538a tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:47.575061 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a9bad65a-2044-43fa-843f-cc4e04ad538a tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:47.576850 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a9bad65a-2044-43fa-843f-cc4e04ad538a tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:47.583529 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a9bad65a-2044-43fa-843f-cc4e04ad538a tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:47.584328 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a9bad65a-2044-43fa-843f-cc4e04ad538a tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:47.584984 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a9bad65a-2044-43fa-843f-cc4e04ad538a tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:47.713859 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-a9bad65a-2044-43fa-843f-cc4e04ad538a tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2aeb52d51d3f44319bb4e114858222f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5515114a058944a49182063213de54d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:47.727939 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-a9bad65a-2044-43fa-843f-cc4e04ad538a tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] 10.222.0.27 "GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae" status: 200 len: 1272 microversion: 2.1 time: 0.195687 Aug 08 02:25:47.728959 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 30/55] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:47 2022] GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae => generated 1272 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:47.750605 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-26aa0c4f-4acb-41d7-beac-aaf2bb01bd82 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:47.763847 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-26aa0c4f-4acb-41d7-beac-aaf2bb01bd82 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:47.784450 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-26aa0c4f-4acb-41d7-beac-aaf2bb01bd82 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:47.785000 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-26aa0c4f-4acb-41d7-beac-aaf2bb01bd82 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:47.785448 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-26aa0c4f-4acb-41d7-beac-aaf2bb01bd82 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:47.791669 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-26aa0c4f-4acb-41d7-beac-aaf2bb01bd82 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:47.792439 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-26aa0c4f-4acb-41d7-beac-aaf2bb01bd82 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:47.792946 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-26aa0c4f-4acb-41d7-beac-aaf2bb01bd82 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:47.894991 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-59207e78-e810-4a2f-9a05-16986d662556 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:47.904064 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-59207e78-e810-4a2f-9a05-16986d662556 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1322 microversion: 2.1 time: 0.779250 Aug 08 02:25:47.907569 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 26/56] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:47 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1322 bytes in 783 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:48.095283 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [req-ebf585e0-d861-4d31-8bc6-9ba1c4bf33dc req-167eced3-8a85-4bfd-baee-d5121bcacbd8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "085a353c-1451-4abc-a9c5-dad9de93fcd4", "tag": "f898d7da-2295-4c47-988e-433dec437c80"}]} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:25:48.124580 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-ebf585e0-d861-4d31-8bc6-9ba1c4bf33dc req-167eced3-8a85-4bfd-baee-d5121bcacbd8 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:48.125175 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-ebf585e0-d861-4d31-8bc6-9ba1c4bf33dc req-167eced3-8a85-4bfd-baee-d5121bcacbd8 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:48.138766 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-26aa0c4f-4acb-41d7-beac-aaf2bb01bd82 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:48.139067 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-ebf585e0-d861-4d31-8bc6-9ba1c4bf33dc req-167eced3-8a85-4bfd-baee-d5121bcacbd8 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:48.146371 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-26aa0c4f-4acb-41d7-beac-aaf2bb01bd82 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1296 microversion: 2.1 time: 0.415708 Aug 08 02:25:48.149741 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 31/57] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:47 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1296 bytes in 420 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:48.170854 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-a5a4bfbe-5a92-4634-a34e-da6f9b4dd69f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:48.172768 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-a5a4bfbe-5a92-4634-a34e-da6f9b4dd69f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:48.191611 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a5a4bfbe-5a92-4634-a34e-da6f9b4dd69f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:48.194797 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a5a4bfbe-5a92-4634-a34e-da6f9b4dd69f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:48.195413 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a5a4bfbe-5a92-4634-a34e-da6f9b4dd69f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:48.198353 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a5a4bfbe-5a92-4634-a34e-da6f9b4dd69f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:48.198811 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a5a4bfbe-5a92-4634-a34e-da6f9b4dd69f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:48.199253 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a5a4bfbe-5a92-4634-a34e-da6f9b4dd69f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:48.214908 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.compute.server_external_events [req-ebf585e0-d861-4d31-8bc6-9ba1c4bf33dc req-167eced3-8a85-4bfd-baee-d5121bcacbd8 service nova] Creating event network-changed:f898d7da-2295-4c47-988e-433dec437c80 for instance 085a353c-1451-4abc-a9c5-dad9de93fcd4 on hv-cinder-74014 Aug 08 02:25:48.215595 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-ebf585e0-d861-4d31-8bc6-9ba1c4bf33dc req-167eced3-8a85-4bfd-baee-d5121bcacbd8 service nova] Acquiring lock "compute-rpcapi-router" {{(pid=118351) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:293}} Aug 08 02:25:48.216107 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-ebf585e0-d861-4d31-8bc6-9ba1c4bf33dc req-167eced3-8a85-4bfd-baee-d5121bcacbd8 service nova] Acquired lock "compute-rpcapi-router" {{(pid=118351) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:296}} Aug 08 02:25:48.234225 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6a38ee93-7049-414c-9b56-9b619783ba0d None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:48.239074 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6a38ee93-7049-414c-9b56-9b619783ba0d None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:48.239633 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6a38ee93-7049-414c-9b56-9b619783ba0d None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:48.240439 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6a38ee93-7049-414c-9b56-9b619783ba0d None None] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:48.246977 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6a38ee93-7049-414c-9b56-9b619783ba0d None None] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:48.248032 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6a38ee93-7049-414c-9b56-9b619783ba0d None None] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:48.294701 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.compute.rpcapi [None req-6a38ee93-7049-414c-9b56-9b619783ba0d None None] Automatically selected compute RPC version 6.0 from minimum service version 61 Aug 08 02:25:48.302513 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6a38ee93-7049-414c-9b56-9b619783ba0d None None] Releasing lock "compute-rpcapi-router" {{(pid=118351) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:314}} Aug 08 02:25:48.351931 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-6a38ee93-7049-414c-9b56-9b619783ba0d None None] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.442670 Aug 08 02:25:48.352565 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 27/58] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:25:47 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 444 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:25:48.429139 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-a5a4bfbe-5a92-4634-a34e-da6f9b4dd69f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:48.442259 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-a5a4bfbe-5a92-4634-a34e-da6f9b4dd69f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:48.465987 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-a5a4bfbe-5a92-4634-a34e-da6f9b4dd69f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:48.465987 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-a5a4bfbe-5a92-4634-a34e-da6f9b4dd69f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1340 microversion: 2.20 time: 0.308859 Aug 08 02:25:48.485292 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 32/59] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:25:48 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1340 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:25:48.558037 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-d198c9b7-5f99-4dd5-bcdd-c1d3d6464e00 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:48.558987 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-d198c9b7-5f99-4dd5-bcdd-c1d3d6464e00 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: 95ae6b45-35d7-43a7-a021-91bb545cac4e] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:48.569619 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d198c9b7-5f99-4dd5-bcdd-c1d3d6464e00 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:48.570392 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d198c9b7-5f99-4dd5-bcdd-c1d3d6464e00 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:48.571112 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d198c9b7-5f99-4dd5-bcdd-c1d3d6464e00 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:48.582903 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d198c9b7-5f99-4dd5-bcdd-c1d3d6464e00 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:48.585532 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d198c9b7-5f99-4dd5-bcdd-c1d3d6464e00 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:48.586239 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d198c9b7-5f99-4dd5-bcdd-c1d3d6464e00 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:48.693655 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-d198c9b7-5f99-4dd5-bcdd-c1d3d6464e00 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:48.709685 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-d198c9b7-5f99-4dd5-bcdd-c1d3d6464e00 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e" status: 200 len: 1299 microversion: 2.1 time: 0.169566 Aug 08 02:25:48.710322 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 28/60] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:48 2022] GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e => generated 1299 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:48.754569 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-acc24cdb-24fe-45bb-bc11-0282e0a84488 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:48.755185 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-acc24cdb-24fe-45bb-bc11-0282e0a84488 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] [instance: 0fc3bfe1-b902-4a65-8c41-81e6f49112ae] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:48.763336 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-acc24cdb-24fe-45bb-bc11-0282e0a84488 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:48.763779 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-acc24cdb-24fe-45bb-bc11-0282e0a84488 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:48.764219 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-acc24cdb-24fe-45bb-bc11-0282e0a84488 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:48.777177 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-acc24cdb-24fe-45bb-bc11-0282e0a84488 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:48.777640 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-acc24cdb-24fe-45bb-bc11-0282e0a84488 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:48.779704 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-acc24cdb-24fe-45bb-bc11-0282e0a84488 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:48.912087 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-acc24cdb-24fe-45bb-bc11-0282e0a84488 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2aeb52d51d3f44319bb4e114858222f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5515114a058944a49182063213de54d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:48.921478 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-acc24cdb-24fe-45bb-bc11-0282e0a84488 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] 10.222.0.27 "GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae" status: 200 len: 1272 microversion: 2.1 time: 0.170891 Aug 08 02:25:48.924526 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 33/61] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:48 2022] GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae => generated 1272 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:48.945261 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-b8e5f9ab-2865-415b-a9ff-9ac633864c5f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:48.947153 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-b8e5f9ab-2865-415b-a9ff-9ac633864c5f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:48.958243 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b8e5f9ab-2865-415b-a9ff-9ac633864c5f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:48.958741 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b8e5f9ab-2865-415b-a9ff-9ac633864c5f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:48.959177 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b8e5f9ab-2865-415b-a9ff-9ac633864c5f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:48.959885 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b8e5f9ab-2865-415b-a9ff-9ac633864c5f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:48.960339 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b8e5f9ab-2865-415b-a9ff-9ac633864c5f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:48.960788 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b8e5f9ab-2865-415b-a9ff-9ac633864c5f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:49.172838 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-8e11a7ea-abd5-4728-89cf-21ad6564a186 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:49.174634 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-8e11a7ea-abd5-4728-89cf-21ad6564a186 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:49.196933 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8e11a7ea-abd5-4728-89cf-21ad6564a186 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:49.201773 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8e11a7ea-abd5-4728-89cf-21ad6564a186 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:49.204897 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8e11a7ea-abd5-4728-89cf-21ad6564a186 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:49.204897 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8e11a7ea-abd5-4728-89cf-21ad6564a186 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:49.204897 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8e11a7ea-abd5-4728-89cf-21ad6564a186 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:49.208873 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8e11a7ea-abd5-4728-89cf-21ad6564a186 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:49.248312 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-b8e5f9ab-2865-415b-a9ff-9ac633864c5f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:49.279386 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-b8e5f9ab-2865-415b-a9ff-9ac633864c5f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1322 microversion: 2.1 time: 0.343011 Aug 08 02:25:49.285181 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 29/62] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:48 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1322 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:49.367084 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-8e11a7ea-abd5-4728-89cf-21ad6564a186 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:49.367084 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-8e11a7ea-abd5-4728-89cf-21ad6564a186 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1296 microversion: 2.1 time: 0.190911 Aug 08 02:25:49.367084 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 34/63] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:49 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1296 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:49.500020 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-dd1c3acb-3a98-4812-acb7-60e857e28b6a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:49.501122 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-dd1c3acb-3a98-4812-acb7-60e857e28b6a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:49.532409 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-dd1c3acb-3a98-4812-acb7-60e857e28b6a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:49.533605 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-dd1c3acb-3a98-4812-acb7-60e857e28b6a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:49.534853 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-dd1c3acb-3a98-4812-acb7-60e857e28b6a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:49.535821 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-dd1c3acb-3a98-4812-acb7-60e857e28b6a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:49.536580 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-dd1c3acb-3a98-4812-acb7-60e857e28b6a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:49.537614 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-dd1c3acb-3a98-4812-acb7-60e857e28b6a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:49.623705 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell1:SELECT=100 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:49.695765 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-dd1c3acb-3a98-4812-acb7-60e857e28b6a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:49.711029 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-dd1c3acb-3a98-4812-acb7-60e857e28b6a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:49.718616 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-dd1c3acb-3a98-4812-acb7-60e857e28b6a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:49.728583 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-dd1c3acb-3a98-4812-acb7-60e857e28b6a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1340 microversion: 2.20 time: 0.229625 Aug 08 02:25:49.730182 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 30/64] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:25:49 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1340 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:25:49.735861 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-bdc58bfe-479e-43c6-b163-5f5470c49551 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:49.737398 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-bdc58bfe-479e-43c6-b163-5f5470c49551 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: 95ae6b45-35d7-43a7-a021-91bb545cac4e] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:49.764438 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-bdc58bfe-479e-43c6-b163-5f5470c49551 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:49.766230 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-bdc58bfe-479e-43c6-b163-5f5470c49551 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:49.767197 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-bdc58bfe-479e-43c6-b163-5f5470c49551 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:49.770665 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-bdc58bfe-479e-43c6-b163-5f5470c49551 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:49.772265 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-bdc58bfe-479e-43c6-b163-5f5470c49551 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:49.777217 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-bdc58bfe-479e-43c6-b163-5f5470c49551 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:49.911898 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-bdc58bfe-479e-43c6-b163-5f5470c49551 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:49.925416 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-bdc58bfe-479e-43c6-b163-5f5470c49551 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e" status: 200 len: 1299 microversion: 2.1 time: 0.193981 Aug 08 02:25:49.930482 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 35/65] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:49 2022] GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e => generated 1299 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:49.949530 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-364fa4e9-6b5d-4774-a8cc-c8d848670ed4 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:49.950151 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-364fa4e9-6b5d-4774-a8cc-c8d848670ed4 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] [instance: 0fc3bfe1-b902-4a65-8c41-81e6f49112ae] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:49.968902 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-364fa4e9-6b5d-4774-a8cc-c8d848670ed4 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:49.969706 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-364fa4e9-6b5d-4774-a8cc-c8d848670ed4 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:49.970326 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-364fa4e9-6b5d-4774-a8cc-c8d848670ed4 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:49.971230 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-364fa4e9-6b5d-4774-a8cc-c8d848670ed4 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:49.971855 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-364fa4e9-6b5d-4774-a8cc-c8d848670ed4 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:49.972451 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-364fa4e9-6b5d-4774-a8cc-c8d848670ed4 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:50.101205 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-364fa4e9-6b5d-4774-a8cc-c8d848670ed4 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2aeb52d51d3f44319bb4e114858222f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5515114a058944a49182063213de54d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:50.140086 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-364fa4e9-6b5d-4774-a8cc-c8d848670ed4 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] 10.222.0.27 "GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae" status: 200 len: 1272 microversion: 2.1 time: 0.200169 Aug 08 02:25:50.150794 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 31/66] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:49 2022] GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae => generated 1272 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:50.337934 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-105747d7-ca66-4629-ad1a-2ac0feddd3c8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:50.339429 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-105747d7-ca66-4629-ad1a-2ac0feddd3c8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:50.361681 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-105747d7-ca66-4629-ad1a-2ac0feddd3c8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:50.365065 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-105747d7-ca66-4629-ad1a-2ac0feddd3c8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:50.366521 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-105747d7-ca66-4629-ad1a-2ac0feddd3c8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:50.372193 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-105747d7-ca66-4629-ad1a-2ac0feddd3c8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:50.376324 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-105747d7-ca66-4629-ad1a-2ac0feddd3c8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:50.379485 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-105747d7-ca66-4629-ad1a-2ac0feddd3c8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:50.402628 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-0f9f63e5-4de6-4da3-a39f-31f5850b2c83 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:50.404057 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-0f9f63e5-4de6-4da3-a39f-31f5850b2c83 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:50.421778 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0f9f63e5-4de6-4da3-a39f-31f5850b2c83 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:50.422282 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0f9f63e5-4de6-4da3-a39f-31f5850b2c83 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:50.422724 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0f9f63e5-4de6-4da3-a39f-31f5850b2c83 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:50.423441 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0f9f63e5-4de6-4da3-a39f-31f5850b2c83 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:50.423895 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0f9f63e5-4de6-4da3-a39f-31f5850b2c83 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:50.444477 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0f9f63e5-4de6-4da3-a39f-31f5850b2c83 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:50.580684 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell0:SELECT=4 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:50.585662 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-0f9f63e5-4de6-4da3-a39f-31f5850b2c83 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:50.603749 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-0f9f63e5-4de6-4da3-a39f-31f5850b2c83 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1296 microversion: 2.1 time: 0.212259 Aug 08 02:25:50.604510 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 32/67] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:50 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1296 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:50.762625 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-29ced88a-98f2-47eb-b9e0-bfe9656d7916 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:50.764169 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-29ced88a-98f2-47eb-b9e0-bfe9656d7916 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:50.775545 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-29ced88a-98f2-47eb-b9e0-bfe9656d7916 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:50.779225 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-29ced88a-98f2-47eb-b9e0-bfe9656d7916 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:50.780562 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-29ced88a-98f2-47eb-b9e0-bfe9656d7916 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:50.787369 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-29ced88a-98f2-47eb-b9e0-bfe9656d7916 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:50.787369 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-29ced88a-98f2-47eb-b9e0-bfe9656d7916 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:50.787369 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-29ced88a-98f2-47eb-b9e0-bfe9656d7916 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:51.277173 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-29ced88a-98f2-47eb-b9e0-bfe9656d7916 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:51.319646 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-29ced88a-98f2-47eb-b9e0-bfe9656d7916 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:51.330058 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-29ced88a-98f2-47eb-b9e0-bfe9656d7916 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:51.332094 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-29ced88a-98f2-47eb-b9e0-bfe9656d7916 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1340 microversion: 2.20 time: 0.573544 Aug 08 02:25:51.334155 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 33/68] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:25:50 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1340 bytes in 576 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 08 02:25:51.348845 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-e2c3ca53-1fb6-4278-a48d-b3f957f5c3f8 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:51.349843 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-e2c3ca53-1fb6-4278-a48d-b3f957f5c3f8 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: 95ae6b45-35d7-43a7-a021-91bb545cac4e] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:51.367825 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e2c3ca53-1fb6-4278-a48d-b3f957f5c3f8 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:51.367825 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e2c3ca53-1fb6-4278-a48d-b3f957f5c3f8 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:51.367825 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e2c3ca53-1fb6-4278-a48d-b3f957f5c3f8 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:51.367825 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e2c3ca53-1fb6-4278-a48d-b3f957f5c3f8 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:51.368223 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e2c3ca53-1fb6-4278-a48d-b3f957f5c3f8 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:51.368223 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e2c3ca53-1fb6-4278-a48d-b3f957f5c3f8 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:51.519020 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-105747d7-ca66-4629-ad1a-2ac0feddd3c8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:51.523609 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell1:SELECT=100 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:51.542412 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-105747d7-ca66-4629-ad1a-2ac0feddd3c8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1322 microversion: 2.1 time: 1.209189 Aug 08 02:25:51.549393 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 36/69] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:50 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1322 bytes in 1212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:51.589715 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-47fa5406-24c5-43d8-8812-9dfddbf068e8 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:51.589715 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-47fa5406-24c5-43d8-8812-9dfddbf068e8 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] [instance: 0fc3bfe1-b902-4a65-8c41-81e6f49112ae] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:51.601213 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-47fa5406-24c5-43d8-8812-9dfddbf068e8 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:51.606072 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-47fa5406-24c5-43d8-8812-9dfddbf068e8 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:51.607707 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-47fa5406-24c5-43d8-8812-9dfddbf068e8 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:51.610145 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-47fa5406-24c5-43d8-8812-9dfddbf068e8 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:51.610887 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-47fa5406-24c5-43d8-8812-9dfddbf068e8 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:51.618468 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-47fa5406-24c5-43d8-8812-9dfddbf068e8 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:51.695399 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell0:SELECT=32 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:51.748317 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell0:SELECT=28 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:52.105016 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-e2c3ca53-1fb6-4278-a48d-b3f957f5c3f8 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:52.113731 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-e2c3ca53-1fb6-4278-a48d-b3f957f5c3f8 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e" status: 200 len: 1366 microversion: 2.1 time: 0.778295 Aug 08 02:25:52.118512 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 34/70] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:51 2022] GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e => generated 1366 bytes in 784 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:52.258467 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-47fa5406-24c5-43d8-8812-9dfddbf068e8 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2aeb52d51d3f44319bb4e114858222f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5515114a058944a49182063213de54d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:52.272943 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-47fa5406-24c5-43d8-8812-9dfddbf068e8 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] 10.222.0.27 "GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae" status: 200 len: 1314 microversion: 2.1 time: 0.706293 Aug 08 02:25:52.274194 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 37/71] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:51 2022] GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae => generated 1314 bytes in 730 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:52.279007 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-decc2ece-c0fa-4eee-95ef-752d09c34903 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:52.284532 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-decc2ece-c0fa-4eee-95ef-752d09c34903 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:52.308376 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-decc2ece-c0fa-4eee-95ef-752d09c34903 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:52.309484 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-decc2ece-c0fa-4eee-95ef-752d09c34903 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:52.311614 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-decc2ece-c0fa-4eee-95ef-752d09c34903 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:52.314981 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-decc2ece-c0fa-4eee-95ef-752d09c34903 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:52.315599 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-decc2ece-c0fa-4eee-95ef-752d09c34903 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:52.318933 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-decc2ece-c0fa-4eee-95ef-752d09c34903 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:52.617220 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [req-ff6d7c09-ba76-4f78-9bc9-d7cc254a0199 req-2861544c-7976-402d-91eb-f9069a1a88b8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ed75f58f-9135-4d97-b24a-2f8fcdab53ce", "tag": "ee2c6536-d687-4d7c-aac6-d3b85f022762"}]} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:25:52.634184 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-ff6d7c09-ba76-4f78-9bc9-d7cc254a0199 req-2861544c-7976-402d-91eb-f9069a1a88b8 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:52.634184 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-ff6d7c09-ba76-4f78-9bc9-d7cc254a0199 req-2861544c-7976-402d-91eb-f9069a1a88b8 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:52.634184 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-ff6d7c09-ba76-4f78-9bc9-d7cc254a0199 req-2861544c-7976-402d-91eb-f9069a1a88b8 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:52.701649 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.compute.server_external_events [req-ff6d7c09-ba76-4f78-9bc9-d7cc254a0199 req-2861544c-7976-402d-91eb-f9069a1a88b8 service nova] Creating event network-changed:ee2c6536-d687-4d7c-aac6-d3b85f022762 for instance ed75f58f-9135-4d97-b24a-2f8fcdab53ce on hv-cinder-74014 Aug 08 02:25:52.738213 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [req-ff6d7c09-ba76-4f78-9bc9-d7cc254a0199 req-2861544c-7976-402d-91eb-f9069a1a88b8 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.615178 Aug 08 02:25:52.739065 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 35/72] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:25:52 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 620 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:25:52.766132 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-0a415cd1-9bc4-4180-8b31-4fa3446f1221 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:52.769984 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-0a415cd1-9bc4-4180-8b31-4fa3446f1221 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:52.770110 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-decc2ece-c0fa-4eee-95ef-752d09c34903 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:52.779223 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-decc2ece-c0fa-4eee-95ef-752d09c34903 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1364 microversion: 2.1 time: 0.503654 Aug 08 02:25:52.783092 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 38/73] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:52 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1364 bytes in 508 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:52.793470 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-d196ef3b-01d5-49d8-9f09-bf4b5d41678c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:52.795572 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-d196ef3b-01d5-49d8-9f09-bf4b5d41678c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:52.796484 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0a415cd1-9bc4-4180-8b31-4fa3446f1221 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:52.797192 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0a415cd1-9bc4-4180-8b31-4fa3446f1221 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:52.797949 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0a415cd1-9bc4-4180-8b31-4fa3446f1221 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:52.802680 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0a415cd1-9bc4-4180-8b31-4fa3446f1221 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:52.802680 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0a415cd1-9bc4-4180-8b31-4fa3446f1221 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:52.802680 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0a415cd1-9bc4-4180-8b31-4fa3446f1221 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:52.814069 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d196ef3b-01d5-49d8-9f09-bf4b5d41678c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:52.814855 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d196ef3b-01d5-49d8-9f09-bf4b5d41678c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:52.817223 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d196ef3b-01d5-49d8-9f09-bf4b5d41678c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:52.818216 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d196ef3b-01d5-49d8-9f09-bf4b5d41678c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:52.820435 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d196ef3b-01d5-49d8-9f09-bf4b5d41678c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:52.821125 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d196ef3b-01d5-49d8-9f09-bf4b5d41678c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:53.596649 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-d196ef3b-01d5-49d8-9f09-bf4b5d41678c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:53.630874 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-d196ef3b-01d5-49d8-9f09-bf4b5d41678c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1322 microversion: 2.1 time: 0.840747 Aug 08 02:25:53.631804 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 39/74] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:52 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1322 bytes in 848 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:53.638343 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-0a415cd1-9bc4-4180-8b31-4fa3446f1221 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:53.655848 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-f934b98f-32a3-408f-9504-fdd1874939c0 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:53.659759 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-f934b98f-32a3-408f-9504-fdd1874939c0 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: 95ae6b45-35d7-43a7-a021-91bb545cac4e] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:53.681020 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-0a415cd1-9bc4-4180-8b31-4fa3446f1221 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:53.685772 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-0a415cd1-9bc4-4180-8b31-4fa3446f1221 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:53.688460 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f934b98f-32a3-408f-9504-fdd1874939c0 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:53.693889 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-0a415cd1-9bc4-4180-8b31-4fa3446f1221 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1408 microversion: 2.20 time: 0.948455 Aug 08 02:25:53.697455 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f934b98f-32a3-408f-9504-fdd1874939c0 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:53.697894 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 36/75] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:25:52 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1408 bytes in 958 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:25:53.699735 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f934b98f-32a3-408f-9504-fdd1874939c0 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:53.702681 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f934b98f-32a3-408f-9504-fdd1874939c0 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:53.703763 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f934b98f-32a3-408f-9504-fdd1874939c0 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:53.705771 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f934b98f-32a3-408f-9504-fdd1874939c0 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:53.719969 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-fd765d5b-fa2d-4c30-985c-840de05506fb tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:53.721296 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-fd765d5b-fa2d-4c30-985c-840de05506fb tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] [instance: 0fc3bfe1-b902-4a65-8c41-81e6f49112ae] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:53.741656 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-fd765d5b-fa2d-4c30-985c-840de05506fb tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:53.742435 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-fd765d5b-fa2d-4c30-985c-840de05506fb tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:53.743991 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-fd765d5b-fa2d-4c30-985c-840de05506fb tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:53.746774 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-fd765d5b-fa2d-4c30-985c-840de05506fb tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:53.747793 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-fd765d5b-fa2d-4c30-985c-840de05506fb tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:53.749482 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-fd765d5b-fa2d-4c30-985c-840de05506fb tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:54.258758 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-fd765d5b-fa2d-4c30-985c-840de05506fb tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2aeb52d51d3f44319bb4e114858222f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5515114a058944a49182063213de54d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:54.276342 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-fd765d5b-fa2d-4c30-985c-840de05506fb tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] 10.222.0.27 "GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae" status: 200 len: 1314 microversion: 2.1 time: 0.567595 Aug 08 02:25:54.277957 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 37/76] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:53 2022] GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae => generated 1314 bytes in 579 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:54.285500 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-1722792d-9485-4b79-b406-f92deca08d95 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:54.287269 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-1722792d-9485-4b79-b406-f92deca08d95 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:54.301290 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1722792d-9485-4b79-b406-f92deca08d95 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:54.313070 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1722792d-9485-4b79-b406-f92deca08d95 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:54.313070 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1722792d-9485-4b79-b406-f92deca08d95 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:54.313070 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1722792d-9485-4b79-b406-f92deca08d95 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:54.313070 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1722792d-9485-4b79-b406-f92deca08d95 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:54.313545 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1722792d-9485-4b79-b406-f92deca08d95 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:54.578024 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-f934b98f-32a3-408f-9504-fdd1874939c0 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:54.589688 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-f934b98f-32a3-408f-9504-fdd1874939c0 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e" status: 200 len: 1366 microversion: 2.1 time: 0.955100 Aug 08 02:25:54.599292 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 40/77] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:53 2022] GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e => generated 1366 bytes in 965 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:54.651786 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-2d712f2c-26cd-4a7c-877b-16cefc822ebc tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:54.652485 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-2d712f2c-26cd-4a7c-877b-16cefc822ebc tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:54.668162 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2d712f2c-26cd-4a7c-877b-16cefc822ebc tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:54.669681 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2d712f2c-26cd-4a7c-877b-16cefc822ebc tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:54.670332 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2d712f2c-26cd-4a7c-877b-16cefc822ebc tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:54.673609 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2d712f2c-26cd-4a7c-877b-16cefc822ebc tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:54.674547 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2d712f2c-26cd-4a7c-877b-16cefc822ebc tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:54.675214 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2d712f2c-26cd-4a7c-877b-16cefc822ebc tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:54.774136 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-1722792d-9485-4b79-b406-f92deca08d95 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:54.789828 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-1722792d-9485-4b79-b406-f92deca08d95 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1364 microversion: 2.1 time: 0.504548 Aug 08 02:25:54.790464 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 38/78] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:54 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1364 bytes in 512 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:54.801597 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-ce8bfb11-5212-42ff-88c2-4960ab11ef49 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:54.802476 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-ce8bfb11-5212-42ff-88c2-4960ab11ef49 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:54.814877 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ce8bfb11-5212-42ff-88c2-4960ab11ef49 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:54.820330 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ce8bfb11-5212-42ff-88c2-4960ab11ef49 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:54.820912 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ce8bfb11-5212-42ff-88c2-4960ab11ef49 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:54.822128 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ce8bfb11-5212-42ff-88c2-4960ab11ef49 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:54.822683 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ce8bfb11-5212-42ff-88c2-4960ab11ef49 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:54.823216 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ce8bfb11-5212-42ff-88c2-4960ab11ef49 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:55.288495 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-ce8bfb11-5212-42ff-88c2-4960ab11ef49 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:55.304748 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-ce8bfb11-5212-42ff-88c2-4960ab11ef49 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:55.306643 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-ce8bfb11-5212-42ff-88c2-4960ab11ef49 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:55.308225 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-ce8bfb11-5212-42ff-88c2-4960ab11ef49 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1408 microversion: 2.20 time: 0.516588 Aug 08 02:25:55.309410 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 39/79] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:25:54 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1408 bytes in 518 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:25:55.317009 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-91bc5f0f-e2ff-43c1-8b80-47e9ad20405f tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:55.318152 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-91bc5f0f-e2ff-43c1-8b80-47e9ad20405f tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] [instance: 0fc3bfe1-b902-4a65-8c41-81e6f49112ae] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:55.327193 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-91bc5f0f-e2ff-43c1-8b80-47e9ad20405f tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:55.327970 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-91bc5f0f-e2ff-43c1-8b80-47e9ad20405f tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:55.329124 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-91bc5f0f-e2ff-43c1-8b80-47e9ad20405f tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:55.330142 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-91bc5f0f-e2ff-43c1-8b80-47e9ad20405f tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:55.330315 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-2d712f2c-26cd-4a7c-877b-16cefc822ebc tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:55.330936 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-91bc5f0f-e2ff-43c1-8b80-47e9ad20405f tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:55.331497 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-91bc5f0f-e2ff-43c1-8b80-47e9ad20405f tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:55.340238 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-2d712f2c-26cd-4a7c-877b-16cefc822ebc tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1322 microversion: 2.1 time: 0.692362 Aug 08 02:25:55.352010 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 41/80] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:54 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1322 bytes in 705 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:55.546068 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-91bc5f0f-e2ff-43c1-8b80-47e9ad20405f tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2aeb52d51d3f44319bb4e114858222f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5515114a058944a49182063213de54d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:55.556517 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-91bc5f0f-e2ff-43c1-8b80-47e9ad20405f tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] 10.222.0.27 "GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae" status: 200 len: 1314 microversion: 2.1 time: 0.244896 Aug 08 02:25:55.560773 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 40/81] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:55 2022] GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae => generated 1314 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:55.612235 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-769839f5-e474-4690-9eaf-a8406d7c03d4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:55.624382 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-769839f5-e474-4690-9eaf-a8406d7c03d4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: 95ae6b45-35d7-43a7-a021-91bb545cac4e] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:55.624382 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-769839f5-e474-4690-9eaf-a8406d7c03d4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:55.624382 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-769839f5-e474-4690-9eaf-a8406d7c03d4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:55.624382 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-769839f5-e474-4690-9eaf-a8406d7c03d4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:55.624851 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-769839f5-e474-4690-9eaf-a8406d7c03d4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:55.624851 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-769839f5-e474-4690-9eaf-a8406d7c03d4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:55.624851 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-769839f5-e474-4690-9eaf-a8406d7c03d4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:55.812751 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-6f82b089-9dda-43a8-9440-2130f4f8c92b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:55.814343 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-6f82b089-9dda-43a8-9440-2130f4f8c92b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:55.820434 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6f82b089-9dda-43a8-9440-2130f4f8c92b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:55.820965 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6f82b089-9dda-43a8-9440-2130f4f8c92b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:55.821487 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6f82b089-9dda-43a8-9440-2130f4f8c92b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:55.822583 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6f82b089-9dda-43a8-9440-2130f4f8c92b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:55.823042 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6f82b089-9dda-43a8-9440-2130f4f8c92b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:55.823459 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6f82b089-9dda-43a8-9440-2130f4f8c92b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:55.824536 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-769839f5-e474-4690-9eaf-a8406d7c03d4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:55.834886 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-769839f5-e474-4690-9eaf-a8406d7c03d4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e" status: 200 len: 1366 microversion: 2.1 time: 0.226522 Aug 08 02:25:55.836786 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 42/82] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:55 2022] GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e => generated 1366 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:55.969764 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-6f82b089-9dda-43a8-9440-2130f4f8c92b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:55.981190 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-6f82b089-9dda-43a8-9440-2130f4f8c92b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1364 microversion: 2.1 time: 0.173862 Aug 08 02:25:55.983522 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 41/83] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:55 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1364 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:25:56.333216 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-5ece9e27-e24e-4ee6-9161-833dc7c0730e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:56.334180 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-5ece9e27-e24e-4ee6-9161-833dc7c0730e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:56.352404 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5ece9e27-e24e-4ee6-9161-833dc7c0730e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:56.353106 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5ece9e27-e24e-4ee6-9161-833dc7c0730e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:56.353761 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5ece9e27-e24e-4ee6-9161-833dc7c0730e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:56.354907 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5ece9e27-e24e-4ee6-9161-833dc7c0730e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:56.356186 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5ece9e27-e24e-4ee6-9161-833dc7c0730e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:56.359633 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5ece9e27-e24e-4ee6-9161-833dc7c0730e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:56.365961 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-54d519a5-37b3-444e-96d3-733d91fd129e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:56.367879 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-54d519a5-37b3-444e-96d3-733d91fd129e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:56.386528 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-54d519a5-37b3-444e-96d3-733d91fd129e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:56.387190 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-54d519a5-37b3-444e-96d3-733d91fd129e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:56.387751 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-54d519a5-37b3-444e-96d3-733d91fd129e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:56.388464 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-54d519a5-37b3-444e-96d3-733d91fd129e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:56.388963 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-54d519a5-37b3-444e-96d3-733d91fd129e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:56.389397 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-54d519a5-37b3-444e-96d3-733d91fd129e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:56.716067 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-5ece9e27-e24e-4ee6-9161-833dc7c0730e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:56.726172 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-5ece9e27-e24e-4ee6-9161-833dc7c0730e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:56.728362 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-5ece9e27-e24e-4ee6-9161-833dc7c0730e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:56.729672 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-5ece9e27-e24e-4ee6-9161-833dc7c0730e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1408 microversion: 2.20 time: 0.401344 Aug 08 02:25:56.730335 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 43/84] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:25:56 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1408 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:25:56.736498 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [req-9514ae1c-b691-4cf1-a841-63764be737bf req-c1a6079b-9611-4e80-a8af-02198116d215 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "95ae6b45-35d7-43a7-a021-91bb545cac4e", "tag": "26e540df-eae8-4a69-873e-d4b1a210cfdf"}]} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:25:56.750604 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-9514ae1c-b691-4cf1-a841-63764be737bf req-c1a6079b-9611-4e80-a8af-02198116d215 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:56.751319 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-9514ae1c-b691-4cf1-a841-63764be737bf req-c1a6079b-9611-4e80-a8af-02198116d215 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:56.752016 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-9514ae1c-b691-4cf1-a841-63764be737bf req-c1a6079b-9611-4e80-a8af-02198116d215 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:56.756631 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-54d519a5-37b3-444e-96d3-733d91fd129e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:56.765468 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-54d519a5-37b3-444e-96d3-733d91fd129e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1322 microversion: 2.1 time: 0.405304 Aug 08 02:25:56.768351 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 42/85] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:56 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1322 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:25:56.786609 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.compute.server_external_events [req-9514ae1c-b691-4cf1-a841-63764be737bf req-c1a6079b-9611-4e80-a8af-02198116d215 service nova] Creating event network-changed:26e540df-eae8-4a69-873e-d4b1a210cfdf for instance 95ae6b45-35d7-43a7-a021-91bb545cac4e on hv-cinder-74014 Aug 08 02:25:56.787200 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-9514ae1c-b691-4cf1-a841-63764be737bf req-c1a6079b-9611-4e80-a8af-02198116d215 service nova] Acquiring lock "compute-rpcapi-router" {{(pid=118352) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:293}} Aug 08 02:25:56.787814 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-9514ae1c-b691-4cf1-a841-63764be737bf req-c1a6079b-9611-4e80-a8af-02198116d215 service nova] Acquired lock "compute-rpcapi-router" {{(pid=118352) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:296}} Aug 08 02:25:56.788944 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-964f9288-e637-402f-a84f-580acef04dfd None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:56.789469 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-964f9288-e637-402f-a84f-580acef04dfd None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:56.789953 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-964f9288-e637-402f-a84f-580acef04dfd None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:56.790843 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-964f9288-e637-402f-a84f-580acef04dfd None None] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:56.791431 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-964f9288-e637-402f-a84f-580acef04dfd None None] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:56.791991 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-964f9288-e637-402f-a84f-580acef04dfd None None] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:56.795811 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-63ccadb6-46bc-409f-9d22-d7e2610d5fe1 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:56.796439 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-63ccadb6-46bc-409f-9d22-d7e2610d5fe1 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] [instance: 0fc3bfe1-b902-4a65-8c41-81e6f49112ae] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:56.801964 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.compute.rpcapi [None req-964f9288-e637-402f-a84f-580acef04dfd None None] Automatically selected compute RPC version 6.0 from minimum service version 61 Aug 08 02:25:56.802841 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-964f9288-e637-402f-a84f-580acef04dfd None None] Releasing lock "compute-rpcapi-router" {{(pid=118352) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:314}} Aug 08 02:25:56.820184 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-964f9288-e637-402f-a84f-580acef04dfd None None] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.088360 Aug 08 02:25:56.821194 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 44/86] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:25:56 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:25:56.823345 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-63ccadb6-46bc-409f-9d22-d7e2610d5fe1 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:56.825195 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-63ccadb6-46bc-409f-9d22-d7e2610d5fe1 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:56.827541 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-63ccadb6-46bc-409f-9d22-d7e2610d5fe1 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:56.830908 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-63ccadb6-46bc-409f-9d22-d7e2610d5fe1 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:56.831413 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-63ccadb6-46bc-409f-9d22-d7e2610d5fe1 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:56.832929 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-63ccadb6-46bc-409f-9d22-d7e2610d5fe1 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:56.858596 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-21924e0f-5e9a-4aae-9606-be09556ceca6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:56.859306 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-21924e0f-5e9a-4aae-9606-be09556ceca6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: 95ae6b45-35d7-43a7-a021-91bb545cac4e] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:56.875802 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-21924e0f-5e9a-4aae-9606-be09556ceca6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:56.876517 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-21924e0f-5e9a-4aae-9606-be09556ceca6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:56.915205 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-21924e0f-5e9a-4aae-9606-be09556ceca6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:56.915205 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-21924e0f-5e9a-4aae-9606-be09556ceca6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:56.915205 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-21924e0f-5e9a-4aae-9606-be09556ceca6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:56.915205 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-21924e0f-5e9a-4aae-9606-be09556ceca6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:57.240423 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-63ccadb6-46bc-409f-9d22-d7e2610d5fe1 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2aeb52d51d3f44319bb4e114858222f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5515114a058944a49182063213de54d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:57.251458 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-63ccadb6-46bc-409f-9d22-d7e2610d5fe1 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] 10.222.0.27 "GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae" status: 200 len: 1314 microversion: 2.1 time: 0.481623 Aug 08 02:25:57.252009 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 43/87] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:56 2022] GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae => generated 1314 bytes in 483 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:57.264823 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-93b9109d-6967-424a-b812-22b296b273f0 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:57.264823 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-93b9109d-6967-424a-b812-22b296b273f0 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:57.265660 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-93b9109d-6967-424a-b812-22b296b273f0 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:57.266307 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-93b9109d-6967-424a-b812-22b296b273f0 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:57.266882 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-93b9109d-6967-424a-b812-22b296b273f0 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:57.267733 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-93b9109d-6967-424a-b812-22b296b273f0 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:57.268378 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-93b9109d-6967-424a-b812-22b296b273f0 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:57.268977 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-93b9109d-6967-424a-b812-22b296b273f0 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:57.279243 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-21924e0f-5e9a-4aae-9606-be09556ceca6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:57.287136 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-21924e0f-5e9a-4aae-9606-be09556ceca6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e" status: 200 len: 1366 microversion: 2.1 time: 0.434085 Aug 08 02:25:57.288092 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 45/88] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:56 2022] GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e => generated 1366 bytes in 436 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:25:57.407253 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-93b9109d-6967-424a-b812-22b296b273f0 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:57.415123 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-93b9109d-6967-424a-b812-22b296b273f0 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1364 microversion: 2.1 time: 0.161919 Aug 08 02:25:57.418282 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 44/89] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:57 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1364 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:57.754363 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-e2bfc9d5-9d74-4ca1-88b2-375ed36dfa57 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:57.755635 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-e2bfc9d5-9d74-4ca1-88b2-375ed36dfa57 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:57.761786 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e2bfc9d5-9d74-4ca1-88b2-375ed36dfa57 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:57.762269 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e2bfc9d5-9d74-4ca1-88b2-375ed36dfa57 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:57.762696 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e2bfc9d5-9d74-4ca1-88b2-375ed36dfa57 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:57.764107 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e2bfc9d5-9d74-4ca1-88b2-375ed36dfa57 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:57.764645 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e2bfc9d5-9d74-4ca1-88b2-375ed36dfa57 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:57.765092 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e2bfc9d5-9d74-4ca1-88b2-375ed36dfa57 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:57.794010 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-d30e75bd-5fac-4254-9794-41e1bcf07923 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:57.794979 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-d30e75bd-5fac-4254-9794-41e1bcf07923 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:57.801399 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d30e75bd-5fac-4254-9794-41e1bcf07923 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:57.801890 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d30e75bd-5fac-4254-9794-41e1bcf07923 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:57.802341 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d30e75bd-5fac-4254-9794-41e1bcf07923 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:57.803155 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d30e75bd-5fac-4254-9794-41e1bcf07923 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:57.803593 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d30e75bd-5fac-4254-9794-41e1bcf07923 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:57.804024 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d30e75bd-5fac-4254-9794-41e1bcf07923 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:58.073197 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-e2bfc9d5-9d74-4ca1-88b2-375ed36dfa57 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:58.090744 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-e2bfc9d5-9d74-4ca1-88b2-375ed36dfa57 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:58.092839 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-e2bfc9d5-9d74-4ca1-88b2-375ed36dfa57 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:58.094855 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-e2bfc9d5-9d74-4ca1-88b2-375ed36dfa57 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1408 microversion: 2.20 time: 0.345080 Aug 08 02:25:58.096167 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 46/90] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:25:57 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1408 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:25:58.125613 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-d30e75bd-5fac-4254-9794-41e1bcf07923 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:58.133525 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-d30e75bd-5fac-4254-9794-41e1bcf07923 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1322 microversion: 2.1 time: 0.343834 Aug 08 02:25:58.134300 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 45/91] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:57 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1322 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:25:58.277832 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-257ba970-f2f3-4520-a69c-3f6abda8e8f9 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:58.278945 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-257ba970-f2f3-4520-a69c-3f6abda8e8f9 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] [instance: 0fc3bfe1-b902-4a65-8c41-81e6f49112ae] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:58.299274 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-257ba970-f2f3-4520-a69c-3f6abda8e8f9 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:58.301799 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-257ba970-f2f3-4520-a69c-3f6abda8e8f9 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:58.303320 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-257ba970-f2f3-4520-a69c-3f6abda8e8f9 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:58.304584 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-257ba970-f2f3-4520-a69c-3f6abda8e8f9 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:58.306345 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-257ba970-f2f3-4520-a69c-3f6abda8e8f9 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:58.322765 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-b5f0789c-9872-481e-ae48-5983fa0dd46b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:58.322765 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-b5f0789c-9872-481e-ae48-5983fa0dd46b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: 95ae6b45-35d7-43a7-a021-91bb545cac4e] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:58.331434 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell0:SELECT=10 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:58.339281 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-257ba970-f2f3-4520-a69c-3f6abda8e8f9 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.033s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:58.341155 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b5f0789c-9872-481e-ae48-5983fa0dd46b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:58.341978 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b5f0789c-9872-481e-ae48-5983fa0dd46b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:58.343347 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b5f0789c-9872-481e-ae48-5983fa0dd46b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:58.344314 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b5f0789c-9872-481e-ae48-5983fa0dd46b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:58.347288 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b5f0789c-9872-481e-ae48-5983fa0dd46b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:58.348032 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b5f0789c-9872-481e-ae48-5983fa0dd46b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:58.683231 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-257ba970-f2f3-4520-a69c-3f6abda8e8f9 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2aeb52d51d3f44319bb4e114858222f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5515114a058944a49182063213de54d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:58.698284 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-257ba970-f2f3-4520-a69c-3f6abda8e8f9 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] 10.222.0.27 "GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae" status: 200 len: 1314 microversion: 2.1 time: 0.425089 Aug 08 02:25:58.698870 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 47/92] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:58 2022] GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae => generated 1314 bytes in 426 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:58.703446 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-b5f0789c-9872-481e-ae48-5983fa0dd46b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:58.709796 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-d2554272-ed93-4f7c-acb1-4af8d50de7b3 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:58.710583 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-d2554272-ed93-4f7c-acb1-4af8d50de7b3 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:58.720921 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-b5f0789c-9872-481e-ae48-5983fa0dd46b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e" status: 200 len: 1366 microversion: 2.1 time: 0.412207 Aug 08 02:25:58.721864 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 46/93] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:58 2022] GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e => generated 1366 bytes in 415 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:58.723788 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d2554272-ed93-4f7c-acb1-4af8d50de7b3 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:58.723788 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d2554272-ed93-4f7c-acb1-4af8d50de7b3 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:58.723788 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d2554272-ed93-4f7c-acb1-4af8d50de7b3 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:58.725574 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d2554272-ed93-4f7c-acb1-4af8d50de7b3 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:58.725574 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d2554272-ed93-4f7c-acb1-4af8d50de7b3 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:58.725574 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d2554272-ed93-4f7c-acb1-4af8d50de7b3 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:58.843942 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [req-d21b82a2-74f9-4553-b4a0-79fbbfa85253 req-5079295a-8567-4918-9146-71f796de2454 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0fc3bfe1-b902-4a65-8c41-81e6f49112ae", "tag": "5482fadc-3651-40bc-b0e6-82d5cf650c16"}, {"name": "network-changed", "server_uuid": "f51a5900-5abc-4188-bbc4-33b0f0f7d091", "tag": "8d2a6aac-ff75-417c-b0b9-cbf799f88d22"}, {"name": "network-changed", "server_uuid": "ddd15bdd-8a1c-455b-91c9-504f69b5308a", "tag": "84fa14fb-cbbd-42d5-a597-bc2556a0ba82"}]} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:25:58.855701 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-d21b82a2-74f9-4553-b4a0-79fbbfa85253 req-5079295a-8567-4918-9146-71f796de2454 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:58.856177 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-d21b82a2-74f9-4553-b4a0-79fbbfa85253 req-5079295a-8567-4918-9146-71f796de2454 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:58.857564 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-d21b82a2-74f9-4553-b4a0-79fbbfa85253 req-5079295a-8567-4918-9146-71f796de2454 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:58.873956 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.compute.server_external_events [req-d21b82a2-74f9-4553-b4a0-79fbbfa85253 req-5079295a-8567-4918-9146-71f796de2454 service nova] Creating event network-changed:5482fadc-3651-40bc-b0e6-82d5cf650c16 for instance 0fc3bfe1-b902-4a65-8c41-81e6f49112ae on hv-cinder-74014 Aug 08 02:25:58.874853 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.compute.server_external_events [req-d21b82a2-74f9-4553-b4a0-79fbbfa85253 req-5079295a-8567-4918-9146-71f796de2454 service nova] Creating event network-changed:8d2a6aac-ff75-417c-b0b9-cbf799f88d22 for instance f51a5900-5abc-4188-bbc4-33b0f0f7d091 on hv-cinder-74014 Aug 08 02:25:58.875600 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.compute.server_external_events [req-d21b82a2-74f9-4553-b4a0-79fbbfa85253 req-5079295a-8567-4918-9146-71f796de2454 service nova] Creating event network-changed:84fa14fb-cbbd-42d5-a597-bc2556a0ba82 for instance ddd15bdd-8a1c-455b-91c9-504f69b5308a on hv-cinder-74014 Aug 08 02:25:58.883652 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [req-d21b82a2-74f9-4553-b4a0-79fbbfa85253 req-5079295a-8567-4918-9146-71f796de2454 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 513 microversion: 2.1 time: 0.045441 Aug 08 02:25:58.884250 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 47/94] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:25:58 2022] POST /compute/v2.1/os-server-external-events => generated 513 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 08 02:25:58.964031 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-d2554272-ed93-4f7c-acb1-4af8d50de7b3 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:58.973000 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-d2554272-ed93-4f7c-acb1-4af8d50de7b3 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1364 microversion: 2.1 time: 0.272214 Aug 08 02:25:58.973786 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 48/95] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:58 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1364 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:59.121162 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-159b2150-278c-48f9-ac52-6b3402c5c654 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:59.127690 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-159b2150-278c-48f9-ac52-6b3402c5c654 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:59.128600 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-159b2150-278c-48f9-ac52-6b3402c5c654 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:59.129121 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-159b2150-278c-48f9-ac52-6b3402c5c654 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:59.129558 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-159b2150-278c-48f9-ac52-6b3402c5c654 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:59.130242 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-159b2150-278c-48f9-ac52-6b3402c5c654 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:59.130697 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-159b2150-278c-48f9-ac52-6b3402c5c654 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:59.131119 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-159b2150-278c-48f9-ac52-6b3402c5c654 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:59.149903 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell1:SELECT=100 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:59.157333 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-694fb72f-8371-4d30-b641-d5133da07431 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:59.158475 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-694fb72f-8371-4d30-b641-d5133da07431 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:59.165216 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-694fb72f-8371-4d30-b641-d5133da07431 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:59.166041 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-694fb72f-8371-4d30-b641-d5133da07431 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:59.166637 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-694fb72f-8371-4d30-b641-d5133da07431 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:59.167555 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-694fb72f-8371-4d30-b641-d5133da07431 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:59.168174 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-694fb72f-8371-4d30-b641-d5133da07431 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:59.169673 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-694fb72f-8371-4d30-b641-d5133da07431 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:59.338944 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-159b2150-278c-48f9-ac52-6b3402c5c654 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:59.348751 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-159b2150-278c-48f9-ac52-6b3402c5c654 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:59.351072 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-159b2150-278c-48f9-ac52-6b3402c5c654 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:59.352385 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-159b2150-278c-48f9-ac52-6b3402c5c654 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1408 microversion: 2.20 time: 0.235611 Aug 08 02:25:59.353608 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 48/96] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:25:59 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1408 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:25:59.372925 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-694fb72f-8371-4d30-b641-d5133da07431 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:25:59.382531 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-694fb72f-8371-4d30-b641-d5133da07431 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1322 microversion: 2.1 time: 0.230501 Aug 08 02:25:59.384144 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 49/97] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:59 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1322 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:25:59.727426 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-16aa5cf7-c69d-4306-a953-00f063ce136c tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:59.728047 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-16aa5cf7-c69d-4306-a953-00f063ce136c tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] [instance: 0fc3bfe1-b902-4a65-8c41-81e6f49112ae] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:59.746109 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-16aa5cf7-c69d-4306-a953-00f063ce136c tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:59.746605 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-16aa5cf7-c69d-4306-a953-00f063ce136c tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:59.747046 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-16aa5cf7-c69d-4306-a953-00f063ce136c tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:59.749110 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-16aa5cf7-c69d-4306-a953-00f063ce136c tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:59.749613 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-16aa5cf7-c69d-4306-a953-00f063ce136c tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:59.756290 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-16aa5cf7-c69d-4306-a953-00f063ce136c tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:59.756629 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-cd75a104-1b88-4f71-ad22-bf3de22e8014 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:25:59.758432 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-cd75a104-1b88-4f71-ad22-bf3de22e8014 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: 95ae6b45-35d7-43a7-a021-91bb545cac4e] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:25:59.780749 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cd75a104-1b88-4f71-ad22-bf3de22e8014 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:59.788296 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cd75a104-1b88-4f71-ad22-bf3de22e8014 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:59.789064 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cd75a104-1b88-4f71-ad22-bf3de22e8014 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:25:59.790483 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cd75a104-1b88-4f71-ad22-bf3de22e8014 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:25:59.791213 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cd75a104-1b88-4f71-ad22-bf3de22e8014 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:25:59.793445 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cd75a104-1b88-4f71-ad22-bf3de22e8014 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:00.071261 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-16aa5cf7-c69d-4306-a953-00f063ce136c tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2aeb52d51d3f44319bb4e114858222f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5515114a058944a49182063213de54d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:00.077729 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-cd75a104-1b88-4f71-ad22-bf3de22e8014 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:00.086424 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-cd75a104-1b88-4f71-ad22-bf3de22e8014 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e" status: 200 len: 1366 microversion: 2.1 time: 0.344665 Aug 08 02:26:00.088566 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 50/98] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:59 2022] GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e => generated 1366 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:00.095364 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-980452e0-3206-443b-a7cc-af55ced423d6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:00.096224 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-980452e0-3206-443b-a7cc-af55ced423d6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:00.103133 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-980452e0-3206-443b-a7cc-af55ced423d6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:00.103874 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-980452e0-3206-443b-a7cc-af55ced423d6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:00.104465 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-980452e0-3206-443b-a7cc-af55ced423d6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:00.105391 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-980452e0-3206-443b-a7cc-af55ced423d6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:00.106028 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-980452e0-3206-443b-a7cc-af55ced423d6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:00.106306 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-16aa5cf7-c69d-4306-a953-00f063ce136c tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] 10.222.0.27 "GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae" status: 200 len: 1314 microversion: 2.1 time: 0.386192 Aug 08 02:26:00.107460 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-980452e0-3206-443b-a7cc-af55ced423d6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:00.107594 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 49/99] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:25:59 2022] GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae => generated 1314 bytes in 388 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:00.255654 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-980452e0-3206-443b-a7cc-af55ced423d6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:00.268654 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-980452e0-3206-443b-a7cc-af55ced423d6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1364 microversion: 2.1 time: 0.178847 Aug 08 02:26:00.270051 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 51/100] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:00 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1364 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:00.376251 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-84c09444-674e-406a-b60e-94da87f2a49d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:00.376906 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-84c09444-674e-406a-b60e-94da87f2a49d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:00.383500 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-84c09444-674e-406a-b60e-94da87f2a49d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:00.384421 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-84c09444-674e-406a-b60e-94da87f2a49d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:00.384897 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-84c09444-674e-406a-b60e-94da87f2a49d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:00.385589 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-84c09444-674e-406a-b60e-94da87f2a49d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:00.386042 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-84c09444-674e-406a-b60e-94da87f2a49d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:00.386462 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-84c09444-674e-406a-b60e-94da87f2a49d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:00.407794 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-44b65848-6d31-4d7a-8a74-03597e3480ab tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:00.408899 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-44b65848-6d31-4d7a-8a74-03597e3480ab tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:00.421611 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-44b65848-6d31-4d7a-8a74-03597e3480ab tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:00.423070 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-44b65848-6d31-4d7a-8a74-03597e3480ab tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:00.423677 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-44b65848-6d31-4d7a-8a74-03597e3480ab tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:00.425317 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-44b65848-6d31-4d7a-8a74-03597e3480ab tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:00.426070 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-44b65848-6d31-4d7a-8a74-03597e3480ab tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:00.426684 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-44b65848-6d31-4d7a-8a74-03597e3480ab tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:00.644700 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-84c09444-674e-406a-b60e-94da87f2a49d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:00.657013 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-84c09444-674e-406a-b60e-94da87f2a49d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:00.659070 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-84c09444-674e-406a-b60e-94da87f2a49d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:00.661332 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-84c09444-674e-406a-b60e-94da87f2a49d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1408 microversion: 2.20 time: 0.288344 Aug 08 02:26:00.662534 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 50/101] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:26:00 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1408 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:26:00.691446 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-44b65848-6d31-4d7a-8a74-03597e3480ab tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:00.703789 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-44b65848-6d31-4d7a-8a74-03597e3480ab tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1322 microversion: 2.1 time: 0.303710 Aug 08 02:26:00.704368 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 52/102] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:00 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1322 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:01.133949 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-4c245974-6bd9-4984-bed5-1f947ec47951 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:01.134561 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-4c245974-6bd9-4984-bed5-1f947ec47951 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: 95ae6b45-35d7-43a7-a021-91bb545cac4e] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:01.137687 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-82b48d39-710d-4a2b-84a7-a4f276b7f62e tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:01.138914 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-82b48d39-710d-4a2b-84a7-a4f276b7f62e tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] [instance: 0fc3bfe1-b902-4a65-8c41-81e6f49112ae] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:01.147784 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-82b48d39-710d-4a2b-84a7-a4f276b7f62e tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:01.148278 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-82b48d39-710d-4a2b-84a7-a4f276b7f62e tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:01.148766 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-82b48d39-710d-4a2b-84a7-a4f276b7f62e tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:01.149561 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-82b48d39-710d-4a2b-84a7-a4f276b7f62e tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:01.149911 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-82b48d39-710d-4a2b-84a7-a4f276b7f62e tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:01.150341 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-82b48d39-710d-4a2b-84a7-a4f276b7f62e tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:01.161480 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4c245974-6bd9-4984-bed5-1f947ec47951 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:01.164009 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4c245974-6bd9-4984-bed5-1f947ec47951 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:01.164009 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4c245974-6bd9-4984-bed5-1f947ec47951 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:01.165440 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4c245974-6bd9-4984-bed5-1f947ec47951 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:01.165910 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4c245974-6bd9-4984-bed5-1f947ec47951 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:01.167509 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4c245974-6bd9-4984-bed5-1f947ec47951 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:01.344008 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-82b48d39-710d-4a2b-84a7-a4f276b7f62e tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2aeb52d51d3f44319bb4e114858222f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5515114a058944a49182063213de54d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:01.353123 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-82b48d39-710d-4a2b-84a7-a4f276b7f62e tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] 10.222.0.27 "GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae" status: 200 len: 1314 microversion: 2.1 time: 0.226600 Aug 08 02:26:01.373538 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 51/103] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:01 2022] GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae => generated 1314 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:01.467938 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-4c245974-6bd9-4984-bed5-1f947ec47951 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:01.471899 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell1:SELECT=100 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:01.480039 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-4c245974-6bd9-4984-bed5-1f947ec47951 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e" status: 200 len: 1366 microversion: 2.1 time: 0.349543 Aug 08 02:26:01.480039 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 53/104] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:01 2022] GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e => generated 1366 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:01.483100 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-6066e142-2e6f-4ce5-bc07-f1f29520ee82 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:01.483807 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-6066e142-2e6f-4ce5-bc07-f1f29520ee82 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:01.490212 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6066e142-2e6f-4ce5-bc07-f1f29520ee82 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:01.490927 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6066e142-2e6f-4ce5-bc07-f1f29520ee82 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:01.491546 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6066e142-2e6f-4ce5-bc07-f1f29520ee82 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:01.492570 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6066e142-2e6f-4ce5-bc07-f1f29520ee82 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:01.493233 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6066e142-2e6f-4ce5-bc07-f1f29520ee82 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:01.493837 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6066e142-2e6f-4ce5-bc07-f1f29520ee82 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:01.703903 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-6066e142-2e6f-4ce5-bc07-f1f29520ee82 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:01.706756 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-6066e142-2e6f-4ce5-bc07-f1f29520ee82 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1364 microversion: 2.1 time: 0.227940 Aug 08 02:26:01.719661 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 54/105] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:01 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1364 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:01.719661 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-3cb47ce4-f5c4-4963-b930-2eaff8a25d82 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:01.719661 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-3cb47ce4-f5c4-4963-b930-2eaff8a25d82 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:01.719661 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3cb47ce4-f5c4-4963-b930-2eaff8a25d82 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:01.721002 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3cb47ce4-f5c4-4963-b930-2eaff8a25d82 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:01.721002 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3cb47ce4-f5c4-4963-b930-2eaff8a25d82 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:01.721608 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3cb47ce4-f5c4-4963-b930-2eaff8a25d82 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:01.722294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3cb47ce4-f5c4-4963-b930-2eaff8a25d82 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:01.722872 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3cb47ce4-f5c4-4963-b930-2eaff8a25d82 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:01.728389 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [req-83ebc787-b3b8-4e67-94d8-9f4b0f81bd3f req-bcdaa7cb-08ee-4ab4-a893-28b15010987f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "085a353c-1451-4abc-a9c5-dad9de93fcd4", "name": "network-vif-plugged", "status": "completed", "tag": "f898d7da-2295-4c47-988e-433dec437c80"}]} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:26:01.738142 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-83ebc787-b3b8-4e67-94d8-9f4b0f81bd3f req-bcdaa7cb-08ee-4ab4-a893-28b15010987f service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:01.738727 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-83ebc787-b3b8-4e67-94d8-9f4b0f81bd3f req-bcdaa7cb-08ee-4ab4-a893-28b15010987f service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:01.739037 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-83ebc787-b3b8-4e67-94d8-9f4b0f81bd3f req-bcdaa7cb-08ee-4ab4-a893-28b15010987f service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:01.751088 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.compute.server_external_events [req-83ebc787-b3b8-4e67-94d8-9f4b0f81bd3f req-bcdaa7cb-08ee-4ab4-a893-28b15010987f service nova] Creating event network-vif-plugged:f898d7da-2295-4c47-988e-433dec437c80 for instance 085a353c-1451-4abc-a9c5-dad9de93fcd4 on hv-cinder-74014 Aug 08 02:26:01.756171 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [req-83ebc787-b3b8-4e67-94d8-9f4b0f81bd3f req-bcdaa7cb-08ee-4ab4-a893-28b15010987f service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.381003 Aug 08 02:26:01.757437 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 52/106] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:26:01 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:26:01.765033 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-0bd97cd7-92a8-4da6-8a1e-2ae4778bf740 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:01.765033 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-0bd97cd7-92a8-4da6-8a1e-2ae4778bf740 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:01.771456 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0bd97cd7-92a8-4da6-8a1e-2ae4778bf740 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:01.772173 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0bd97cd7-92a8-4da6-8a1e-2ae4778bf740 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:01.772604 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0bd97cd7-92a8-4da6-8a1e-2ae4778bf740 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:01.773321 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0bd97cd7-92a8-4da6-8a1e-2ae4778bf740 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:01.773926 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0bd97cd7-92a8-4da6-8a1e-2ae4778bf740 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:01.774500 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0bd97cd7-92a8-4da6-8a1e-2ae4778bf740 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:01.949163 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-3cb47ce4-f5c4-4963-b930-2eaff8a25d82 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:01.960369 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-3cb47ce4-f5c4-4963-b930-2eaff8a25d82 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:01.962070 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-3cb47ce4-f5c4-4963-b930-2eaff8a25d82 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:01.963544 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-3cb47ce4-f5c4-4963-b930-2eaff8a25d82 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1408 microversion: 2.20 time: 0.255136 Aug 08 02:26:01.964566 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 55/107] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:26:01 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1408 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:26:01.982802 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-0bd97cd7-92a8-4da6-8a1e-2ae4778bf740 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:01.990276 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-0bd97cd7-92a8-4da6-8a1e-2ae4778bf740 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1322 microversion: 2.1 time: 0.231499 Aug 08 02:26:01.990845 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 53/108] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:01 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1322 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:02.394068 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-be7ccba4-0049-4e97-bbce-a077ac47b182 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:02.394969 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-be7ccba4-0049-4e97-bbce-a077ac47b182 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] [instance: 0fc3bfe1-b902-4a65-8c41-81e6f49112ae] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:02.405761 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-be7ccba4-0049-4e97-bbce-a077ac47b182 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:02.406371 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-be7ccba4-0049-4e97-bbce-a077ac47b182 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:02.406949 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-be7ccba4-0049-4e97-bbce-a077ac47b182 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:02.412387 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-be7ccba4-0049-4e97-bbce-a077ac47b182 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:02.412387 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-be7ccba4-0049-4e97-bbce-a077ac47b182 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:02.412387 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-be7ccba4-0049-4e97-bbce-a077ac47b182 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:02.497048 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-f9ab3446-2a5e-4818-8478-cbdf8046aefc tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:02.498971 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-f9ab3446-2a5e-4818-8478-cbdf8046aefc tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: 95ae6b45-35d7-43a7-a021-91bb545cac4e] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:02.510829 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f9ab3446-2a5e-4818-8478-cbdf8046aefc tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:02.511569 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f9ab3446-2a5e-4818-8478-cbdf8046aefc tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:02.512155 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f9ab3446-2a5e-4818-8478-cbdf8046aefc tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:02.513054 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f9ab3446-2a5e-4818-8478-cbdf8046aefc tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:02.514285 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f9ab3446-2a5e-4818-8478-cbdf8046aefc tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:02.514855 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f9ab3446-2a5e-4818-8478-cbdf8046aefc tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:02.631909 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-be7ccba4-0049-4e97-bbce-a077ac47b182 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2aeb52d51d3f44319bb4e114858222f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5515114a058944a49182063213de54d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:02.661075 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-be7ccba4-0049-4e97-bbce-a077ac47b182 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] 10.222.0.27 "GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae" status: 200 len: 1314 microversion: 2.1 time: 0.265389 Aug 08 02:26:02.661075 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 56/109] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:02 2022] GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae => generated 1314 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:02.732263 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-10747428-fa0d-486a-afab-0b65effc274c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:02.732922 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-10747428-fa0d-486a-afab-0b65effc274c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:02.741232 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-10747428-fa0d-486a-afab-0b65effc274c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:02.742119 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-10747428-fa0d-486a-afab-0b65effc274c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:02.742773 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-10747428-fa0d-486a-afab-0b65effc274c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:02.744454 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-10747428-fa0d-486a-afab-0b65effc274c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:02.745029 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-10747428-fa0d-486a-afab-0b65effc274c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:02.745905 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-10747428-fa0d-486a-afab-0b65effc274c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:02.774604 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-f9ab3446-2a5e-4818-8478-cbdf8046aefc tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:02.784631 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-f9ab3446-2a5e-4818-8478-cbdf8046aefc tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e" status: 200 len: 1366 microversion: 2.1 time: 0.290581 Aug 08 02:26:02.784631 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 54/110] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:02 2022] GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e => generated 1366 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:02.953344 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-10747428-fa0d-486a-afab-0b65effc274c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:02.968031 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-10747428-fa0d-486a-afab-0b65effc274c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1364 microversion: 2.1 time: 0.240170 Aug 08 02:26:02.969250 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 57/111] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:02 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1364 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:26:02.989505 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-dc535e04-0797-486b-bcec-9c496cbce34d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:02.990161 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-dc535e04-0797-486b-bcec-9c496cbce34d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:02.996487 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-dc535e04-0797-486b-bcec-9c496cbce34d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:02.997012 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-dc535e04-0797-486b-bcec-9c496cbce34d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:02.998014 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-dc535e04-0797-486b-bcec-9c496cbce34d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:02.998889 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-dc535e04-0797-486b-bcec-9c496cbce34d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:02.999559 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-dc535e04-0797-486b-bcec-9c496cbce34d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:03.002582 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-dc535e04-0797-486b-bcec-9c496cbce34d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:03.012220 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-72845b43-325e-4f5b-92ee-bacd9f3b7b28 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:03.013084 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-72845b43-325e-4f5b-92ee-bacd9f3b7b28 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:03.019716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-72845b43-325e-4f5b-92ee-bacd9f3b7b28 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:03.024427 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-72845b43-325e-4f5b-92ee-bacd9f3b7b28 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:03.024993 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-72845b43-325e-4f5b-92ee-bacd9f3b7b28 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:03.025809 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-72845b43-325e-4f5b-92ee-bacd9f3b7b28 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:03.026373 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-72845b43-325e-4f5b-92ee-bacd9f3b7b28 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:03.026942 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-72845b43-325e-4f5b-92ee-bacd9f3b7b28 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:03.230146 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-dc535e04-0797-486b-bcec-9c496cbce34d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:03.244609 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-dc535e04-0797-486b-bcec-9c496cbce34d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:03.248877 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-dc535e04-0797-486b-bcec-9c496cbce34d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:03.248877 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-dc535e04-0797-486b-bcec-9c496cbce34d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1408 microversion: 2.20 time: 0.262450 Aug 08 02:26:03.262979 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-72845b43-325e-4f5b-92ee-bacd9f3b7b28 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:03.264650 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 55/112] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:26:02 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1408 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:26:03.272277 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-72845b43-325e-4f5b-92ee-bacd9f3b7b28 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1322 microversion: 2.1 time: 0.264540 Aug 08 02:26:03.273430 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 58/113] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:03 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1322 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:26:03.690647 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-db9ca396-15e6-4e63-a552-6bc74e95654d tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:03.694150 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-db9ca396-15e6-4e63-a552-6bc74e95654d tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] [instance: 0fc3bfe1-b902-4a65-8c41-81e6f49112ae] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:03.699061 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-db9ca396-15e6-4e63-a552-6bc74e95654d tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:03.699747 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-db9ca396-15e6-4e63-a552-6bc74e95654d tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:03.701006 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-db9ca396-15e6-4e63-a552-6bc74e95654d tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:03.701909 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-db9ca396-15e6-4e63-a552-6bc74e95654d tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:03.702337 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-db9ca396-15e6-4e63-a552-6bc74e95654d tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:03.702926 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-db9ca396-15e6-4e63-a552-6bc74e95654d tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:03.775549 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [req-f50be0fa-187d-47eb-ba79-7d1903873ed2 req-96cf1d58-0b24-4669-bf83-96587ea86e11 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ed75f58f-9135-4d97-b24a-2f8fcdab53ce", "name": "network-vif-plugged", "status": "completed", "tag": "ee2c6536-d687-4d7c-aac6-d3b85f022762"}]} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:26:03.804370 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-f50be0fa-187d-47eb-ba79-7d1903873ed2 req-96cf1d58-0b24-4669-bf83-96587ea86e11 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:03.805970 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-f50be0fa-187d-47eb-ba79-7d1903873ed2 req-96cf1d58-0b24-4669-bf83-96587ea86e11 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:03.807696 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-f50be0fa-187d-47eb-ba79-7d1903873ed2 req-96cf1d58-0b24-4669-bf83-96587ea86e11 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:03.837929 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.compute.server_external_events [req-f50be0fa-187d-47eb-ba79-7d1903873ed2 req-96cf1d58-0b24-4669-bf83-96587ea86e11 service nova] Creating event network-vif-plugged:ee2c6536-d687-4d7c-aac6-d3b85f022762 for instance ed75f58f-9135-4d97-b24a-2f8fcdab53ce on hv-cinder-74014 Aug 08 02:26:03.839702 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [req-f50be0fa-187d-47eb-ba79-7d1903873ed2 req-96cf1d58-0b24-4669-bf83-96587ea86e11 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.069518 Aug 08 02:26:03.841337 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 59/114] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:26:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 08 02:26:03.848349 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-acdc3b82-1c61-4c7d-93b9-981f8870b95e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:03.849180 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-acdc3b82-1c61-4c7d-93b9-981f8870b95e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: 95ae6b45-35d7-43a7-a021-91bb545cac4e] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:03.856951 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-acdc3b82-1c61-4c7d-93b9-981f8870b95e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:03.857949 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-acdc3b82-1c61-4c7d-93b9-981f8870b95e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:03.858750 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-acdc3b82-1c61-4c7d-93b9-981f8870b95e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:03.859640 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-acdc3b82-1c61-4c7d-93b9-981f8870b95e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:03.860278 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-acdc3b82-1c61-4c7d-93b9-981f8870b95e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:03.860920 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-acdc3b82-1c61-4c7d-93b9-981f8870b95e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:03.926457 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-db9ca396-15e6-4e63-a552-6bc74e95654d tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2aeb52d51d3f44319bb4e114858222f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5515114a058944a49182063213de54d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:03.934128 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-db9ca396-15e6-4e63-a552-6bc74e95654d tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] 10.222.0.27 "GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae" status: 200 len: 1314 microversion: 2.1 time: 0.254457 Aug 08 02:26:03.935230 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 56/115] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:03 2022] GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae => generated 1314 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:04.009398 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-2d452f0d-e3f5-4993-ad72-54e58c484150 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:04.010009 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-2d452f0d-e3f5-4993-ad72-54e58c484150 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:04.016615 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2d452f0d-e3f5-4993-ad72-54e58c484150 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:04.017138 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2d452f0d-e3f5-4993-ad72-54e58c484150 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:04.017573 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2d452f0d-e3f5-4993-ad72-54e58c484150 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:04.018265 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2d452f0d-e3f5-4993-ad72-54e58c484150 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:04.018719 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2d452f0d-e3f5-4993-ad72-54e58c484150 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:04.019140 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2d452f0d-e3f5-4993-ad72-54e58c484150 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:04.024639 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-acdc3b82-1c61-4c7d-93b9-981f8870b95e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:04.032388 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-acdc3b82-1c61-4c7d-93b9-981f8870b95e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e" status: 200 len: 1366 microversion: 2.1 time: 0.188662 Aug 08 02:26:04.033078 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 60/116] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:03 2022] GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e => generated 1366 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:04.153708 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-2d452f0d-e3f5-4993-ad72-54e58c484150 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:04.162428 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-2d452f0d-e3f5-4993-ad72-54e58c484150 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1364 microversion: 2.1 time: 0.156755 Aug 08 02:26:04.163322 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 57/117] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:04 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1364 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:26:04.270573 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-da0370cf-827a-442a-9fe4-b130ca1613d0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:04.271185 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-da0370cf-827a-442a-9fe4-b130ca1613d0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:04.277251 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-da0370cf-827a-442a-9fe4-b130ca1613d0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:04.277728 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-da0370cf-827a-442a-9fe4-b130ca1613d0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:04.278159 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-da0370cf-827a-442a-9fe4-b130ca1613d0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:04.278875 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-da0370cf-827a-442a-9fe4-b130ca1613d0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:04.279504 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-da0370cf-827a-442a-9fe4-b130ca1613d0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:04.279930 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-da0370cf-827a-442a-9fe4-b130ca1613d0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:04.295765 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-078c6f28-dc44-4ef3-9638-9535a0cc0773 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:04.297013 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-078c6f28-dc44-4ef3-9638-9535a0cc0773 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:04.303962 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-078c6f28-dc44-4ef3-9638-9535a0cc0773 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:04.304452 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-078c6f28-dc44-4ef3-9638-9535a0cc0773 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:04.305034 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-078c6f28-dc44-4ef3-9638-9535a0cc0773 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:04.305885 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-078c6f28-dc44-4ef3-9638-9535a0cc0773 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:04.306336 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-078c6f28-dc44-4ef3-9638-9535a0cc0773 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:04.307005 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-078c6f28-dc44-4ef3-9638-9535a0cc0773 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:04.498059 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-da0370cf-827a-442a-9fe4-b130ca1613d0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:04.506061 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-da0370cf-827a-442a-9fe4-b130ca1613d0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:04.507581 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-da0370cf-827a-442a-9fe4-b130ca1613d0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:04.508707 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-da0370cf-827a-442a-9fe4-b130ca1613d0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1408 microversion: 2.20 time: 0.242013 Aug 08 02:26:04.509267 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 61/118] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:26:04 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1408 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:26:04.509506 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-078c6f28-dc44-4ef3-9638-9535a0cc0773 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:04.517819 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-078c6f28-dc44-4ef3-9638-9535a0cc0773 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1322 microversion: 2.1 time: 0.226559 Aug 08 02:26:04.520702 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 58/119] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:04 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1322 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:26:04.957685 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-7d17e2e7-9718-4a9c-9fee-9b88748410ee tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:04.958225 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-7d17e2e7-9718-4a9c-9fee-9b88748410ee tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] [instance: 0fc3bfe1-b902-4a65-8c41-81e6f49112ae] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:04.964110 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7d17e2e7-9718-4a9c-9fee-9b88748410ee tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:04.964588 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7d17e2e7-9718-4a9c-9fee-9b88748410ee tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:04.965050 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7d17e2e7-9718-4a9c-9fee-9b88748410ee tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:04.965726 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7d17e2e7-9718-4a9c-9fee-9b88748410ee tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:04.966212 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7d17e2e7-9718-4a9c-9fee-9b88748410ee tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:04.966632 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7d17e2e7-9718-4a9c-9fee-9b88748410ee tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:05.053818 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-3ca696f7-6786-4393-be78-abd6ed534d91 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:05.054637 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-3ca696f7-6786-4393-be78-abd6ed534d91 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: 95ae6b45-35d7-43a7-a021-91bb545cac4e] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:05.061587 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3ca696f7-6786-4393-be78-abd6ed534d91 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:05.062070 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3ca696f7-6786-4393-be78-abd6ed534d91 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:05.062499 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3ca696f7-6786-4393-be78-abd6ed534d91 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:05.063189 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3ca696f7-6786-4393-be78-abd6ed534d91 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:05.063639 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3ca696f7-6786-4393-be78-abd6ed534d91 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:05.064063 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3ca696f7-6786-4393-be78-abd6ed534d91 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:05.111274 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-7d17e2e7-9718-4a9c-9fee-9b88748410ee tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2aeb52d51d3f44319bb4e114858222f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5515114a058944a49182063213de54d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:05.118911 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-7d17e2e7-9718-4a9c-9fee-9b88748410ee tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] 10.222.0.27 "GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae" status: 200 len: 1314 microversion: 2.1 time: 0.165116 Aug 08 02:26:05.119493 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 62/120] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:04 2022] GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae => generated 1314 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:05.183190 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-ae3f7e39-0feb-429b-9800-d933047114d9 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:05.183889 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-ae3f7e39-0feb-429b-9800-d933047114d9 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:05.190934 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ae3f7e39-0feb-429b-9800-d933047114d9 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:05.194862 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ae3f7e39-0feb-429b-9800-d933047114d9 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:05.194862 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ae3f7e39-0feb-429b-9800-d933047114d9 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:05.194862 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ae3f7e39-0feb-429b-9800-d933047114d9 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:05.194862 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ae3f7e39-0feb-429b-9800-d933047114d9 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:05.195171 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ae3f7e39-0feb-429b-9800-d933047114d9 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:05.197849 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-3ca696f7-6786-4393-be78-abd6ed534d91 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:05.205434 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-3ca696f7-6786-4393-be78-abd6ed534d91 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e" status: 200 len: 1366 microversion: 2.1 time: 0.155385 Aug 08 02:26:05.206009 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 59/121] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:05 2022] GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e => generated 1366 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:26:05.344610 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-ae3f7e39-0feb-429b-9800-d933047114d9 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:05.354108 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-ae3f7e39-0feb-429b-9800-d933047114d9 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1364 microversion: 2.1 time: 0.174572 Aug 08 02:26:05.355400 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 63/122] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:05 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1364 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:26:05.537184 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-6efa2041-477f-4d90-bf2c-d11b6ccd2e91 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:05.538021 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-6efa2041-477f-4d90-bf2c-d11b6ccd2e91 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:05.539684 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-091a5fee-ba48-493d-9791-74e55ef90d42 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:05.540301 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-091a5fee-ba48-493d-9791-74e55ef90d42 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:05.547456 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6efa2041-477f-4d90-bf2c-d11b6ccd2e91 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:05.548901 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6efa2041-477f-4d90-bf2c-d11b6ccd2e91 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:05.550004 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6efa2041-477f-4d90-bf2c-d11b6ccd2e91 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:05.550287 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-091a5fee-ba48-493d-9791-74e55ef90d42 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:05.550785 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-091a5fee-ba48-493d-9791-74e55ef90d42 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:05.551355 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6efa2041-477f-4d90-bf2c-d11b6ccd2e91 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:05.552115 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6efa2041-477f-4d90-bf2c-d11b6ccd2e91 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:05.552806 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6efa2041-477f-4d90-bf2c-d11b6ccd2e91 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:05.554078 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-091a5fee-ba48-493d-9791-74e55ef90d42 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:05.554842 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-091a5fee-ba48-493d-9791-74e55ef90d42 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:05.555589 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-091a5fee-ba48-493d-9791-74e55ef90d42 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:05.555713 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-091a5fee-ba48-493d-9791-74e55ef90d42 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:05.853407 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-6efa2041-477f-4d90-bf2c-d11b6ccd2e91 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:05.856520 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-091a5fee-ba48-493d-9791-74e55ef90d42 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:05.862632 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-6efa2041-477f-4d90-bf2c-d11b6ccd2e91 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:05.865209 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-091a5fee-ba48-493d-9791-74e55ef90d42 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1515 microversion: 2.1 time: 0.329915 Aug 08 02:26:05.880607 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 64/123] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:05 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1515 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:26:05.880607 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [req-7a908d94-5d49-42e5-9244-bd7f40852112 req-31871136-1575-4701-bf54-236201e7c2cb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ddd15bdd-8a1c-455b-91c9-504f69b5308a", "name": "network-vif-plugged", "status": "completed", "tag": "84fa14fb-cbbd-42d5-a597-bc2556a0ba82"}]} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:26:05.882089 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-7a908d94-5d49-42e5-9244-bd7f40852112 req-31871136-1575-4701-bf54-236201e7c2cb service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:05.882653 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-7a908d94-5d49-42e5-9244-bd7f40852112 req-31871136-1575-4701-bf54-236201e7c2cb service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:05.883121 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-7a908d94-5d49-42e5-9244-bd7f40852112 req-31871136-1575-4701-bf54-236201e7c2cb service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:05.885633 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-6efa2041-477f-4d90-bf2c-d11b6ccd2e91 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:05.889261 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-6efa2041-477f-4d90-bf2c-d11b6ccd2e91 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1408 microversion: 2.20 time: 0.355501 Aug 08 02:26:05.890301 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 60/124] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:26:05 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1408 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:26:05.895921 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-08feb046-d52e-4786-86d7-c20dff838e92 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:26:05.897958 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-08feb046-d52e-4786-86d7-c20dff838e92 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:05.905459 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-08feb046-d52e-4786-86d7-c20dff838e92 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:05.906023 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-08feb046-d52e-4786-86d7-c20dff838e92 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:05.906574 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-08feb046-d52e-4786-86d7-c20dff838e92 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:05.907405 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-08feb046-d52e-4786-86d7-c20dff838e92 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:05.907930 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-08feb046-d52e-4786-86d7-c20dff838e92 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:05.908481 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-08feb046-d52e-4786-86d7-c20dff838e92 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:05.912312 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.compute.server_external_events [req-7a908d94-5d49-42e5-9244-bd7f40852112 req-31871136-1575-4701-bf54-236201e7c2cb service nova] Creating event network-vif-plugged:84fa14fb-cbbd-42d5-a597-bc2556a0ba82 for instance ddd15bdd-8a1c-455b-91c9-504f69b5308a on hv-cinder-74014 Aug 08 02:26:05.918038 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [req-7a908d94-5d49-42e5-9244-bd7f40852112 req-31871136-1575-4701-bf54-236201e7c2cb service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.050741 Aug 08 02:26:05.918836 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 65/125] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:26:05 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:26:05.978305 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.objects.instance [None req-08feb046-d52e-4786-86d7-c20dff838e92 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lazy-loading 'flavor' on Instance uuid 085a353c-1451-4abc-a9c5-dad9de93fcd4 {{(pid=118351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 08 02:26:06.141492 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-5e37cc89-274e-4f33-a130-4e9d4c70b277 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:06.142272 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-5e37cc89-274e-4f33-a130-4e9d4c70b277 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] [instance: 0fc3bfe1-b902-4a65-8c41-81e6f49112ae] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:06.148680 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5e37cc89-274e-4f33-a130-4e9d4c70b277 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:06.149229 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5e37cc89-274e-4f33-a130-4e9d4c70b277 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:06.149668 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5e37cc89-274e-4f33-a130-4e9d4c70b277 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:06.150363 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5e37cc89-274e-4f33-a130-4e9d4c70b277 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:06.150815 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5e37cc89-274e-4f33-a130-4e9d4c70b277 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:06.151239 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5e37cc89-274e-4f33-a130-4e9d4c70b277 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:06.306783 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-5e37cc89-274e-4f33-a130-4e9d4c70b277 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2aeb52d51d3f44319bb4e114858222f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5515114a058944a49182063213de54d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:06.315911 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-5e37cc89-274e-4f33-a130-4e9d4c70b277 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] 10.222.0.27 "GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae" status: 200 len: 1314 microversion: 2.1 time: 0.180275 Aug 08 02:26:06.316499 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 66/126] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:06 2022] GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae => generated 1314 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:26:06.321185 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-f5b617d0-99f1-45e3-98f3-21d61c10f2cb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:06.323103 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-f5b617d0-99f1-45e3-98f3-21d61c10f2cb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: 95ae6b45-35d7-43a7-a021-91bb545cac4e] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:06.328070 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f5b617d0-99f1-45e3-98f3-21d61c10f2cb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:06.328564 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f5b617d0-99f1-45e3-98f3-21d61c10f2cb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:06.329115 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f5b617d0-99f1-45e3-98f3-21d61c10f2cb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:06.329813 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f5b617d0-99f1-45e3-98f3-21d61c10f2cb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:06.330269 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f5b617d0-99f1-45e3-98f3-21d61c10f2cb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:06.330695 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f5b617d0-99f1-45e3-98f3-21d61c10f2cb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:06.386551 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-08feb046-d52e-4786-86d7-c20dff838e92 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "POST /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4/action" status: 200 len: 32 microversion: 2.1 time: 0.494858 Aug 08 02:26:06.388027 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 61/127] 10.222.0.27 () {60 vars in 1278 bytes} [Mon Aug 8 02:26:05 2022] POST /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4/action => generated 32 bytes in 497 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Aug 08 02:26:06.401868 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-d168c3cb-1ae7-46c9-b297-f69070ef59e5 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:06.404604 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-d168c3cb-1ae7-46c9-b297-f69070ef59e5 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:06.419438 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d168c3cb-1ae7-46c9-b297-f69070ef59e5 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:06.422353 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d168c3cb-1ae7-46c9-b297-f69070ef59e5 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:06.422353 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d168c3cb-1ae7-46c9-b297-f69070ef59e5 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:06.422353 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d168c3cb-1ae7-46c9-b297-f69070ef59e5 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:06.422353 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d168c3cb-1ae7-46c9-b297-f69070ef59e5 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:06.423486 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d168c3cb-1ae7-46c9-b297-f69070ef59e5 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:06.537264 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-f5b617d0-99f1-45e3-98f3-21d61c10f2cb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:06.546346 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-f5b617d0-99f1-45e3-98f3-21d61c10f2cb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e" status: 200 len: 1366 microversion: 2.1 time: 0.228774 Aug 08 02:26:06.546934 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 67/128] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:06 2022] GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e => generated 1366 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:06.569629 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-5d580f0d-15f9-433f-afce-3c470fa9335c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:06.570242 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-5d580f0d-15f9-433f-afce-3c470fa9335c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:06.576799 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5d580f0d-15f9-433f-afce-3c470fa9335c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:06.579705 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5d580f0d-15f9-433f-afce-3c470fa9335c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:06.588795 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5d580f0d-15f9-433f-afce-3c470fa9335c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:06.589791 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5d580f0d-15f9-433f-afce-3c470fa9335c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:06.590292 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5d580f0d-15f9-433f-afce-3c470fa9335c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:06.593855 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5d580f0d-15f9-433f-afce-3c470fa9335c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:06.718917 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-d168c3cb-1ae7-46c9-b297-f69070ef59e5 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:06.729592 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-d168c3cb-1ae7-46c9-b297-f69070ef59e5 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1364 microversion: 2.1 time: 0.339994 Aug 08 02:26:06.730184 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 62/129] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:06 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1364 bytes in 341 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:06.799589 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell0:SELECT=2 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:06.843498 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-5d580f0d-15f9-433f-afce-3c470fa9335c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:06.854853 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-5d580f0d-15f9-433f-afce-3c470fa9335c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1521 microversion: 2.1 time: 0.289078 Aug 08 02:26:06.854853 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 68/130] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:06 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1521 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:06.913210 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-93339aec-65a9-4605-8aef-62674daba37b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:06.913813 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-93339aec-65a9-4605-8aef-62674daba37b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:06.920588 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-93339aec-65a9-4605-8aef-62674daba37b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:06.921089 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-93339aec-65a9-4605-8aef-62674daba37b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:06.921513 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-93339aec-65a9-4605-8aef-62674daba37b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:06.922192 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-93339aec-65a9-4605-8aef-62674daba37b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:06.922640 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-93339aec-65a9-4605-8aef-62674daba37b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:06.923063 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-93339aec-65a9-4605-8aef-62674daba37b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:07.070203 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-93339aec-65a9-4605-8aef-62674daba37b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:07.074074 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:07.079573 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-93339aec-65a9-4605-8aef-62674daba37b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:07.081400 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-93339aec-65a9-4605-8aef-62674daba37b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:07.082765 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-93339aec-65a9-4605-8aef-62674daba37b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1408 microversion: 2.20 time: 0.173460 Aug 08 02:26:07.083373 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 63/131] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:26:06 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1408 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 08 02:26:07.338178 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-f34dc735-ea95-4fc2-997f-5ae58923370d tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:07.338797 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-f34dc735-ea95-4fc2-997f-5ae58923370d tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] [instance: 0fc3bfe1-b902-4a65-8c41-81e6f49112ae] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:07.344877 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f34dc735-ea95-4fc2-997f-5ae58923370d tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:07.345369 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f34dc735-ea95-4fc2-997f-5ae58923370d tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:07.345810 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f34dc735-ea95-4fc2-997f-5ae58923370d tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:07.346495 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f34dc735-ea95-4fc2-997f-5ae58923370d tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:07.346950 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f34dc735-ea95-4fc2-997f-5ae58923370d tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:07.347574 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f34dc735-ea95-4fc2-997f-5ae58923370d tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:07.568472 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-eef6e9fe-edf3-4463-ba20-19b68b10a7c4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:07.569320 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-eef6e9fe-edf3-4463-ba20-19b68b10a7c4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: 95ae6b45-35d7-43a7-a021-91bb545cac4e] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:07.580942 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-eef6e9fe-edf3-4463-ba20-19b68b10a7c4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:07.581781 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-eef6e9fe-edf3-4463-ba20-19b68b10a7c4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:07.582523 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-eef6e9fe-edf3-4463-ba20-19b68b10a7c4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:07.583632 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-eef6e9fe-edf3-4463-ba20-19b68b10a7c4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:07.584263 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-eef6e9fe-edf3-4463-ba20-19b68b10a7c4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:07.584922 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-eef6e9fe-edf3-4463-ba20-19b68b10a7c4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:07.671945 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-f34dc735-ea95-4fc2-997f-5ae58923370d tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2aeb52d51d3f44319bb4e114858222f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5515114a058944a49182063213de54d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:07.690085 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-f34dc735-ea95-4fc2-997f-5ae58923370d tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] 10.222.0.27 "GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae" status: 200 len: 1314 microversion: 2.1 time: 0.355849 Aug 08 02:26:07.690688 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 69/132] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:07 2022] GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae => generated 1314 bytes in 357 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:26:07.752498 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-c58bf786-f2d6-46a2-bc1d-9ce26ab23064 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:07.753354 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-c58bf786-f2d6-46a2-bc1d-9ce26ab23064 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:07.759744 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c58bf786-f2d6-46a2-bc1d-9ce26ab23064 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:07.761974 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c58bf786-f2d6-46a2-bc1d-9ce26ab23064 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:07.761974 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c58bf786-f2d6-46a2-bc1d-9ce26ab23064 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:07.761974 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c58bf786-f2d6-46a2-bc1d-9ce26ab23064 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:07.761974 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c58bf786-f2d6-46a2-bc1d-9ce26ab23064 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:07.762452 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c58bf786-f2d6-46a2-bc1d-9ce26ab23064 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:07.791274 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-eef6e9fe-edf3-4463-ba20-19b68b10a7c4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:07.799412 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-eef6e9fe-edf3-4463-ba20-19b68b10a7c4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e" status: 200 len: 1366 microversion: 2.1 time: 0.235626 Aug 08 02:26:07.805120 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 64/133] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:07 2022] GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e => generated 1366 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:07.876497 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-c2ecf200-a410-4db6-a4bb-4a8d7758f536 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:07.877185 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-c2ecf200-a410-4db6-a4bb-4a8d7758f536 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:07.883084 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c2ecf200-a410-4db6-a4bb-4a8d7758f536 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:07.883563 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c2ecf200-a410-4db6-a4bb-4a8d7758f536 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:07.884186 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c2ecf200-a410-4db6-a4bb-4a8d7758f536 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:07.885060 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c2ecf200-a410-4db6-a4bb-4a8d7758f536 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:07.888222 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c2ecf200-a410-4db6-a4bb-4a8d7758f536 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:07.888222 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c2ecf200-a410-4db6-a4bb-4a8d7758f536 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:07.920600 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-c58bf786-f2d6-46a2-bc1d-9ce26ab23064 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:07.928860 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-c58bf786-f2d6-46a2-bc1d-9ce26ab23064 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1364 microversion: 2.1 time: 0.180525 Aug 08 02:26:07.929798 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 70/134] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:07 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1364 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:26:07.978710 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [req-b97fba19-4b2e-4007-a9ef-6b02837eb15b req-9a6c4c52-6b8d-4381-9e3a-0d593eaacdb8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0fc3bfe1-b902-4a65-8c41-81e6f49112ae", "name": "network-vif-plugged", "status": "completed", "tag": "5482fadc-3651-40bc-b0e6-82d5cf650c16"}]} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:26:07.990749 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-b97fba19-4b2e-4007-a9ef-6b02837eb15b req-9a6c4c52-6b8d-4381-9e3a-0d593eaacdb8 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:07.991313 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-b97fba19-4b2e-4007-a9ef-6b02837eb15b req-9a6c4c52-6b8d-4381-9e3a-0d593eaacdb8 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:07.992267 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-b97fba19-4b2e-4007-a9ef-6b02837eb15b req-9a6c4c52-6b8d-4381-9e3a-0d593eaacdb8 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:08.009471 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.compute.server_external_events [req-b97fba19-4b2e-4007-a9ef-6b02837eb15b req-9a6c4c52-6b8d-4381-9e3a-0d593eaacdb8 service nova] Creating event network-vif-plugged:5482fadc-3651-40bc-b0e6-82d5cf650c16 for instance 0fc3bfe1-b902-4a65-8c41-81e6f49112ae on hv-cinder-74014 Aug 08 02:26:08.016561 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [req-b97fba19-4b2e-4007-a9ef-6b02837eb15b req-9a6c4c52-6b8d-4381-9e3a-0d593eaacdb8 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041927 Aug 08 02:26:08.017541 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 71/135] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:26:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 08 02:26:08.046459 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-c2ecf200-a410-4db6-a4bb-4a8d7758f536 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:08.054424 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-c2ecf200-a410-4db6-a4bb-4a8d7758f536 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1521 microversion: 2.1 time: 0.181794 Aug 08 02:26:08.055002 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 65/136] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:07 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1521 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:26:08.108906 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-f736d771-bdb1-4817-b708-76f016eefe28 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:08.109603 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-f736d771-bdb1-4817-b708-76f016eefe28 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:08.116191 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f736d771-bdb1-4817-b708-76f016eefe28 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:08.116699 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f736d771-bdb1-4817-b708-76f016eefe28 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:08.117471 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f736d771-bdb1-4817-b708-76f016eefe28 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:08.117815 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f736d771-bdb1-4817-b708-76f016eefe28 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:08.118264 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f736d771-bdb1-4817-b708-76f016eefe28 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:08.118680 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f736d771-bdb1-4817-b708-76f016eefe28 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:08.138818 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell1:SELECT=100 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:08.258884 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-f736d771-bdb1-4817-b708-76f016eefe28 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:08.267497 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-f736d771-bdb1-4817-b708-76f016eefe28 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:08.269328 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-f736d771-bdb1-4817-b708-76f016eefe28 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:08.272719 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-f736d771-bdb1-4817-b708-76f016eefe28 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1408 microversion: 2.20 time: 0.167310 Aug 08 02:26:08.272719 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 72/137] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:26:08 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1408 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:26:08.711495 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-a3ea0d32-1d2f-4be2-86a6-fab64da0f6e1 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:08.712104 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-a3ea0d32-1d2f-4be2-86a6-fab64da0f6e1 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] [instance: 0fc3bfe1-b902-4a65-8c41-81e6f49112ae] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:08.718875 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a3ea0d32-1d2f-4be2-86a6-fab64da0f6e1 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:08.719359 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a3ea0d32-1d2f-4be2-86a6-fab64da0f6e1 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:08.719787 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a3ea0d32-1d2f-4be2-86a6-fab64da0f6e1 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:08.720489 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a3ea0d32-1d2f-4be2-86a6-fab64da0f6e1 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:08.720970 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a3ea0d32-1d2f-4be2-86a6-fab64da0f6e1 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:08.721397 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a3ea0d32-1d2f-4be2-86a6-fab64da0f6e1 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:08.821967 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-c0b66ded-9c25-407a-bdc2-2b5269d36c3f tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:08.823328 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-c0b66ded-9c25-407a-bdc2-2b5269d36c3f tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: 95ae6b45-35d7-43a7-a021-91bb545cac4e] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:08.851253 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c0b66ded-9c25-407a-bdc2-2b5269d36c3f tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:08.851886 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c0b66ded-9c25-407a-bdc2-2b5269d36c3f tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:08.852203 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c0b66ded-9c25-407a-bdc2-2b5269d36c3f tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:08.852927 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c0b66ded-9c25-407a-bdc2-2b5269d36c3f tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:08.853382 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c0b66ded-9c25-407a-bdc2-2b5269d36c3f tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:08.864679 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c0b66ded-9c25-407a-bdc2-2b5269d36c3f tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:08.917912 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-a3ea0d32-1d2f-4be2-86a6-fab64da0f6e1 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2aeb52d51d3f44319bb4e114858222f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5515114a058944a49182063213de54d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:08.927806 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-a3ea0d32-1d2f-4be2-86a6-fab64da0f6e1 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] 10.222.0.27 "GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae" status: 200 len: 1314 microversion: 2.1 time: 0.219988 Aug 08 02:26:08.929097 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 66/138] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:08 2022] GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae => generated 1314 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:08.967064 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-42cde279-fb64-404d-9ff4-b73556307615 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:08.968601 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-42cde279-fb64-404d-9ff4-b73556307615 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:08.982958 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-42cde279-fb64-404d-9ff4-b73556307615 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:08.984197 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-42cde279-fb64-404d-9ff4-b73556307615 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:08.985713 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-42cde279-fb64-404d-9ff4-b73556307615 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:08.988289 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-42cde279-fb64-404d-9ff4-b73556307615 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:08.989640 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-42cde279-fb64-404d-9ff4-b73556307615 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:08.990359 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-42cde279-fb64-404d-9ff4-b73556307615 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:09.167680 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-c0b66ded-9c25-407a-bdc2-2b5269d36c3f tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:09.176052 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-c0b66ded-9c25-407a-bdc2-2b5269d36c3f tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e" status: 200 len: 1366 microversion: 2.1 time: 0.358462 Aug 08 02:26:09.176693 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 73/139] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:08 2022] GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e => generated 1366 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:09.190851 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-025373c3-54b3-400b-a27c-e2d5c2a8ec37 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:09.190851 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-025373c3-54b3-400b-a27c-e2d5c2a8ec37 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:09.190851 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-025373c3-54b3-400b-a27c-e2d5c2a8ec37 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:09.190851 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-025373c3-54b3-400b-a27c-e2d5c2a8ec37 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:09.193695 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-025373c3-54b3-400b-a27c-e2d5c2a8ec37 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:09.193695 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-025373c3-54b3-400b-a27c-e2d5c2a8ec37 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:09.193695 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-025373c3-54b3-400b-a27c-e2d5c2a8ec37 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:09.193695 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-025373c3-54b3-400b-a27c-e2d5c2a8ec37 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:09.223677 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-42cde279-fb64-404d-9ff4-b73556307615 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:09.232692 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-42cde279-fb64-404d-9ff4-b73556307615 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1551 microversion: 2.1 time: 0.276585 Aug 08 02:26:09.233690 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 67/140] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:08 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1551 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:09.255911 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-8ba67fe4-1cf8-469e-9704-28a362450293 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:09.256544 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-8ba67fe4-1cf8-469e-9704-28a362450293 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:09.265995 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8ba67fe4-1cf8-469e-9704-28a362450293 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:09.266613 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8ba67fe4-1cf8-469e-9704-28a362450293 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:09.268442 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8ba67fe4-1cf8-469e-9704-28a362450293 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:09.270351 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8ba67fe4-1cf8-469e-9704-28a362450293 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:09.270840 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8ba67fe4-1cf8-469e-9704-28a362450293 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:09.273349 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8ba67fe4-1cf8-469e-9704-28a362450293 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:09.391045 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-025373c3-54b3-400b-a27c-e2d5c2a8ec37 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:09.399650 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-025373c3-54b3-400b-a27c-e2d5c2a8ec37 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1521 microversion: 2.1 time: 0.221848 Aug 08 02:26:09.400833 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 74/141] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:09 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1521 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:09.411179 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-8ba67fe4-1cf8-469e-9704-28a362450293 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.160058 Aug 08 02:26:09.418033 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 68/142] 10.222.0.27 () {58 vars in 1275 bytes} [Mon Aug 8 02:26:09 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a/os-interface => generated 285 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 08 02:26:09.423120 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-3cea155e-5fee-451e-a3c1-44a5e4698558 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:09.445657 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-3cea155e-5fee-451e-a3c1-44a5e4698558 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:09.468398 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3cea155e-5fee-451e-a3c1-44a5e4698558 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:09.468955 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3cea155e-5fee-451e-a3c1-44a5e4698558 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:09.469401 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3cea155e-5fee-451e-a3c1-44a5e4698558 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:09.470115 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3cea155e-5fee-451e-a3c1-44a5e4698558 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:09.476301 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3cea155e-5fee-451e-a3c1-44a5e4698558 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:09.476808 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3cea155e-5fee-451e-a3c1-44a5e4698558 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:09.718015 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-3cea155e-5fee-451e-a3c1-44a5e4698558 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:09.726812 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-3cea155e-5fee-451e-a3c1-44a5e4698558 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:09.729141 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-3cea155e-5fee-451e-a3c1-44a5e4698558 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:09.730342 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-3cea155e-5fee-451e-a3c1-44a5e4698558 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1408 microversion: 2.20 time: 0.328367 Aug 08 02:26:09.730972 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 75/143] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:26:09 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1408 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:26:09.952611 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-210c7812-9013-44bd-bcf0-b3e2a709b7e0 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:09.953297 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-210c7812-9013-44bd-bcf0-b3e2a709b7e0 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] [instance: 0fc3bfe1-b902-4a65-8c41-81e6f49112ae] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:09.959820 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-210c7812-9013-44bd-bcf0-b3e2a709b7e0 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:09.960509 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-210c7812-9013-44bd-bcf0-b3e2a709b7e0 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:09.961005 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-210c7812-9013-44bd-bcf0-b3e2a709b7e0 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:09.961675 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-210c7812-9013-44bd-bcf0-b3e2a709b7e0 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:09.962124 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-210c7812-9013-44bd-bcf0-b3e2a709b7e0 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:09.962757 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-210c7812-9013-44bd-bcf0-b3e2a709b7e0 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:10.036197 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [req-db0c92dd-8de0-4460-9ecd-c1fed09b5458 req-5b1bfe96-39b7-4109-9e8f-7f6f19155918 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f51a5900-5abc-4188-bbc4-33b0f0f7d091", "name": "network-vif-plugged", "status": "completed", "tag": "8d2a6aac-ff75-417c-b0b9-cbf799f88d22"}, {"server_uuid": "95ae6b45-35d7-43a7-a021-91bb545cac4e", "name": "network-vif-plugged", "status": "completed", "tag": "26e540df-eae8-4a69-873e-d4b1a210cfdf"}]} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:26:10.047674 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-db0c92dd-8de0-4460-9ecd-c1fed09b5458 req-5b1bfe96-39b7-4109-9e8f-7f6f19155918 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:10.048389 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-db0c92dd-8de0-4460-9ecd-c1fed09b5458 req-5b1bfe96-39b7-4109-9e8f-7f6f19155918 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:10.049002 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-db0c92dd-8de0-4460-9ecd-c1fed09b5458 req-5b1bfe96-39b7-4109-9e8f-7f6f19155918 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:10.070164 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.compute.server_external_events [req-db0c92dd-8de0-4460-9ecd-c1fed09b5458 req-5b1bfe96-39b7-4109-9e8f-7f6f19155918 service nova] Creating event network-vif-plugged:8d2a6aac-ff75-417c-b0b9-cbf799f88d22 for instance f51a5900-5abc-4188-bbc4-33b0f0f7d091 on hv-cinder-74014 Aug 08 02:26:10.070681 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.compute.server_external_events [req-db0c92dd-8de0-4460-9ecd-c1fed09b5458 req-5b1bfe96-39b7-4109-9e8f-7f6f19155918 service nova] Creating event network-vif-plugged:26e540df-eae8-4a69-873e-d4b1a210cfdf for instance 95ae6b45-35d7-43a7-a021-91bb545cac4e on hv-cinder-74014 Aug 08 02:26:10.078480 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [req-db0c92dd-8de0-4460-9ecd-c1fed09b5458 req-5b1bfe96-39b7-4109-9e8f-7f6f19155918 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.046108 Aug 08 02:26:10.079078 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 76/144] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:26:10 2022] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 08 02:26:10.114868 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-210c7812-9013-44bd-bcf0-b3e2a709b7e0 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2aeb52d51d3f44319bb4e114858222f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5515114a058944a49182063213de54d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:10.123504 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-210c7812-9013-44bd-bcf0-b3e2a709b7e0 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] 10.222.0.27 "GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae" status: 200 len: 1314 microversion: 2.1 time: 0.175834 Aug 08 02:26:10.137328 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 69/145] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:09 2022] GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae => generated 1314 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:10.202239 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-bc9cabe8-575f-42a7-b2eb-c22050f47c31 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:10.202994 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-bc9cabe8-575f-42a7-b2eb-c22050f47c31 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: 95ae6b45-35d7-43a7-a021-91bb545cac4e] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:10.206191 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_api:SELECT=100 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:10.214025 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-bc9cabe8-575f-42a7-b2eb-c22050f47c31 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:10.215204 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-bc9cabe8-575f-42a7-b2eb-c22050f47c31 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:10.215583 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-bc9cabe8-575f-42a7-b2eb-c22050f47c31 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:10.217327 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-bc9cabe8-575f-42a7-b2eb-c22050f47c31 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:10.217452 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-bc9cabe8-575f-42a7-b2eb-c22050f47c31 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:10.217879 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-bc9cabe8-575f-42a7-b2eb-c22050f47c31 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:10.369042 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-bc9cabe8-575f-42a7-b2eb-c22050f47c31 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:10.378063 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-bc9cabe8-575f-42a7-b2eb-c22050f47c31 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e" status: 200 len: 1366 microversion: 2.1 time: 0.183089 Aug 08 02:26:10.378750 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 77/146] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:10 2022] GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e => generated 1366 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:10.431759 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-4f3d1fcf-094f-4258-a80b-a7638b1d6381 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:10.432518 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-4f3d1fcf-094f-4258-a80b-a7638b1d6381 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:10.441626 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4f3d1fcf-094f-4258-a80b-a7638b1d6381 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:10.442121 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4f3d1fcf-094f-4258-a80b-a7638b1d6381 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:10.442558 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4f3d1fcf-094f-4258-a80b-a7638b1d6381 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:10.443257 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4f3d1fcf-094f-4258-a80b-a7638b1d6381 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:10.443711 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4f3d1fcf-094f-4258-a80b-a7638b1d6381 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:10.444157 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4f3d1fcf-094f-4258-a80b-a7638b1d6381 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:10.607753 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-4f3d1fcf-094f-4258-a80b-a7638b1d6381 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:10.616139 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-4f3d1fcf-094f-4258-a80b-a7638b1d6381 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1521 microversion: 2.1 time: 0.190466 Aug 08 02:26:10.620716 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 70/147] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:10 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1521 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:10.753767 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-204427ca-e889-4fae-b2cd-4a91bfffd213 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:10.754644 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-204427ca-e889-4fae-b2cd-4a91bfffd213 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:10.761910 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-204427ca-e889-4fae-b2cd-4a91bfffd213 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:10.762403 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-204427ca-e889-4fae-b2cd-4a91bfffd213 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:10.763689 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-204427ca-e889-4fae-b2cd-4a91bfffd213 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:10.764405 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-204427ca-e889-4fae-b2cd-4a91bfffd213 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:10.765606 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-204427ca-e889-4fae-b2cd-4a91bfffd213 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:10.766269 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-204427ca-e889-4fae-b2cd-4a91bfffd213 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:11.052768 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-204427ca-e889-4fae-b2cd-4a91bfffd213 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:11.066363 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-204427ca-e889-4fae-b2cd-4a91bfffd213 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:11.072707 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-204427ca-e889-4fae-b2cd-4a91bfffd213 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:11.076327 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-204427ca-e889-4fae-b2cd-4a91bfffd213 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1408 microversion: 2.20 time: 0.325359 Aug 08 02:26:11.092703 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 78/148] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:26:10 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1408 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:26:11.144541 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-8797f1f0-0ecf-4f3d-a34f-9bb829d5ae17 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:11.145162 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-8797f1f0-0ecf-4f3d-a34f-9bb829d5ae17 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] [instance: 0fc3bfe1-b902-4a65-8c41-81e6f49112ae] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:11.158567 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8797f1f0-0ecf-4f3d-a34f-9bb829d5ae17 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:11.161334 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8797f1f0-0ecf-4f3d-a34f-9bb829d5ae17 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:11.162981 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8797f1f0-0ecf-4f3d-a34f-9bb829d5ae17 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:11.168225 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8797f1f0-0ecf-4f3d-a34f-9bb829d5ae17 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:11.173728 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8797f1f0-0ecf-4f3d-a34f-9bb829d5ae17 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:11.176034 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8797f1f0-0ecf-4f3d-a34f-9bb829d5ae17 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:11.273556 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [req-2891f1cb-acc1-4235-8241-ab0b6e4034f9 req-4358bb91-d035-4e48-b57b-3b30715973ca service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ddd15bdd-8a1c-455b-91c9-504f69b5308a", "tag": "84fa14fb-cbbd-42d5-a597-bc2556a0ba82"}]} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:26:11.321690 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-2891f1cb-acc1-4235-8241-ab0b6e4034f9 req-4358bb91-d035-4e48-b57b-3b30715973ca service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:11.322178 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-2891f1cb-acc1-4235-8241-ab0b6e4034f9 req-4358bb91-d035-4e48-b57b-3b30715973ca service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:11.322615 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-2891f1cb-acc1-4235-8241-ab0b6e4034f9 req-4358bb91-d035-4e48-b57b-3b30715973ca service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:11.338118 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.compute.server_external_events [req-2891f1cb-acc1-4235-8241-ab0b6e4034f9 req-4358bb91-d035-4e48-b57b-3b30715973ca service nova] Creating event network-changed:84fa14fb-cbbd-42d5-a597-bc2556a0ba82 for instance ddd15bdd-8a1c-455b-91c9-504f69b5308a on hv-cinder-74014 Aug 08 02:26:11.347382 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [req-2891f1cb-acc1-4235-8241-ab0b6e4034f9 req-4358bb91-d035-4e48-b57b-3b30715973ca service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.080789 Aug 08 02:26:11.349445 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 79/149] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:26:11 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 08 02:26:11.378583 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-8797f1f0-0ecf-4f3d-a34f-9bb829d5ae17 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2aeb52d51d3f44319bb4e114858222f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5515114a058944a49182063213de54d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:11.386088 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-8797f1f0-0ecf-4f3d-a34f-9bb829d5ae17 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] 10.222.0.27 "GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae" status: 200 len: 1501 microversion: 2.1 time: 0.245405 Aug 08 02:26:11.386658 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 71/150] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:11 2022] GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae => generated 1501 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:11.401593 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-73d734c7-03b3-40bd-aa43-ebc5402b5360 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:11.402532 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-73d734c7-03b3-40bd-aa43-ebc5402b5360 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: 95ae6b45-35d7-43a7-a021-91bb545cac4e] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:11.411052 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-73d734c7-03b3-40bd-aa43-ebc5402b5360 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:11.411909 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-73d734c7-03b3-40bd-aa43-ebc5402b5360 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:11.412904 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-73d734c7-03b3-40bd-aa43-ebc5402b5360 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:11.415279 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-73d734c7-03b3-40bd-aa43-ebc5402b5360 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:11.415279 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-73d734c7-03b3-40bd-aa43-ebc5402b5360 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:11.415279 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-73d734c7-03b3-40bd-aa43-ebc5402b5360 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:11.597764 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-73d734c7-03b3-40bd-aa43-ebc5402b5360 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:11.604796 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-73d734c7-03b3-40bd-aa43-ebc5402b5360 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e" status: 200 len: 1366 microversion: 2.1 time: 0.208936 Aug 08 02:26:11.607141 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 80/151] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:11 2022] GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e => generated 1366 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:11.638829 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-6a15d928-29f5-4478-aae1-df38dafa8124 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:11.640305 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-6a15d928-29f5-4478-aae1-df38dafa8124 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:11.647211 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6a15d928-29f5-4478-aae1-df38dafa8124 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:11.647872 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6a15d928-29f5-4478-aae1-df38dafa8124 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:11.648484 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6a15d928-29f5-4478-aae1-df38dafa8124 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:11.649395 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6a15d928-29f5-4478-aae1-df38dafa8124 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:11.650121 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6a15d928-29f5-4478-aae1-df38dafa8124 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:11.650672 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6a15d928-29f5-4478-aae1-df38dafa8124 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:11.857949 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-6a15d928-29f5-4478-aae1-df38dafa8124 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:11.863161 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-6a15d928-29f5-4478-aae1-df38dafa8124 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1521 microversion: 2.1 time: 0.229308 Aug 08 02:26:11.863951 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 72/152] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:11 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1521 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:12.101512 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-125b3fd6-ae12-4ed7-beb7-3805acf4cf5c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:12.102112 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-125b3fd6-ae12-4ed7-beb7-3805acf4cf5c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:12.112189 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-125b3fd6-ae12-4ed7-beb7-3805acf4cf5c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:12.112189 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-125b3fd6-ae12-4ed7-beb7-3805acf4cf5c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:12.112189 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-125b3fd6-ae12-4ed7-beb7-3805acf4cf5c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:12.112189 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-125b3fd6-ae12-4ed7-beb7-3805acf4cf5c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:12.112793 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-125b3fd6-ae12-4ed7-beb7-3805acf4cf5c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:12.112793 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-125b3fd6-ae12-4ed7-beb7-3805acf4cf5c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:12.297630 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-125b3fd6-ae12-4ed7-beb7-3805acf4cf5c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:12.317531 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-125b3fd6-ae12-4ed7-beb7-3805acf4cf5c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:12.322637 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-125b3fd6-ae12-4ed7-beb7-3805acf4cf5c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:12.323782 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-125b3fd6-ae12-4ed7-beb7-3805acf4cf5c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1408 microversion: 2.20 time: 0.226174 Aug 08 02:26:12.360834 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 81/153] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:26:12 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1408 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:26:12.628132 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-63dd279c-47ee-41e3-887d-be60bfff2954 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:12.628722 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-63dd279c-47ee-41e3-887d-be60bfff2954 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: 95ae6b45-35d7-43a7-a021-91bb545cac4e] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:12.635182 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-63dd279c-47ee-41e3-887d-be60bfff2954 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:12.635842 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-63dd279c-47ee-41e3-887d-be60bfff2954 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:12.636554 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-63dd279c-47ee-41e3-887d-be60bfff2954 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:12.637275 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-63dd279c-47ee-41e3-887d-be60bfff2954 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:12.638070 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-63dd279c-47ee-41e3-887d-be60bfff2954 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:12.638679 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-63dd279c-47ee-41e3-887d-be60bfff2954 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:12.803545 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-63dd279c-47ee-41e3-887d-be60bfff2954 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:12.815105 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-63dd279c-47ee-41e3-887d-be60bfff2954 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e" status: 200 len: 1366 microversion: 2.1 time: 0.188298 Aug 08 02:26:12.815105 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 73/154] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:12 2022] GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e => generated 1366 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:12.885699 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-fb4605fe-60b1-4ee4-b00a-815c7e9c0c06 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:12.887939 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-fb4605fe-60b1-4ee4-b00a-815c7e9c0c06 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:12.895728 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-fb4605fe-60b1-4ee4-b00a-815c7e9c0c06 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:12.896213 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-fb4605fe-60b1-4ee4-b00a-815c7e9c0c06 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:12.896784 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-fb4605fe-60b1-4ee4-b00a-815c7e9c0c06 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:12.897516 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-fb4605fe-60b1-4ee4-b00a-815c7e9c0c06 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:12.899154 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-fb4605fe-60b1-4ee4-b00a-815c7e9c0c06 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:12.899595 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-fb4605fe-60b1-4ee4-b00a-815c7e9c0c06 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:13.031040 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-fb4605fe-60b1-4ee4-b00a-815c7e9c0c06 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:13.039039 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-fb4605fe-60b1-4ee4-b00a-815c7e9c0c06 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1521 microversion: 2.1 time: 0.158320 Aug 08 02:26:13.039976 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 82/155] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:12 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1521 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:13.373245 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-2d5386e8-ef03-43ed-a40b-26ecfd8207b9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:13.374901 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-2d5386e8-ef03-43ed-a40b-26ecfd8207b9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:13.380987 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2d5386e8-ef03-43ed-a40b-26ecfd8207b9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:13.385846 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2d5386e8-ef03-43ed-a40b-26ecfd8207b9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:13.385846 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2d5386e8-ef03-43ed-a40b-26ecfd8207b9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:13.385846 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2d5386e8-ef03-43ed-a40b-26ecfd8207b9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:13.385846 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2d5386e8-ef03-43ed-a40b-26ecfd8207b9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:13.386302 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2d5386e8-ef03-43ed-a40b-26ecfd8207b9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:13.514824 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-70294653-a015-4079-80c2-95a0819ed76f tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "3362e01d-3da6-4ba8-8efd-92d49164e4eb"}} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:26:13.518484 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-70294653-a015-4079-80c2-95a0819ed76f tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] [instance: 0fc3bfe1-b902-4a65-8c41-81e6f49112ae] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:13.524952 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-70294653-a015-4079-80c2-95a0819ed76f tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:13.525642 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-70294653-a015-4079-80c2-95a0819ed76f tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:13.533625 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-70294653-a015-4079-80c2-95a0819ed76f tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:13.533625 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-70294653-a015-4079-80c2-95a0819ed76f tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:13.533625 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-70294653-a015-4079-80c2-95a0819ed76f tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:13.533625 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-70294653-a015-4079-80c2-95a0819ed76f tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:13.598937 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-2d5386e8-ef03-43ed-a40b-26ecfd8207b9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:13.608413 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-2d5386e8-ef03-43ed-a40b-26ecfd8207b9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:13.609973 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-2d5386e8-ef03-43ed-a40b-26ecfd8207b9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:13.611140 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-2d5386e8-ef03-43ed-a40b-26ecfd8207b9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1600 microversion: 2.20 time: 0.242433 Aug 08 02:26:13.611707 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 74/156] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:26:13 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1600 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:26:13.636304 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-989131ab-b469-493d-a9ef-aba15178d471 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:13.637466 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-989131ab-b469-493d-a9ef-aba15178d471 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:13.645119 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-989131ab-b469-493d-a9ef-aba15178d471 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:13.645915 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-989131ab-b469-493d-a9ef-aba15178d471 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:13.646353 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-989131ab-b469-493d-a9ef-aba15178d471 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:13.647046 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-989131ab-b469-493d-a9ef-aba15178d471 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:13.647491 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-989131ab-b469-493d-a9ef-aba15178d471 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:13.647923 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-989131ab-b469-493d-a9ef-aba15178d471 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:13.745678 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-989131ab-b469-493d-a9ef-aba15178d471 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091/os-interface" status: 200 len: 284 microversion: 2.20 time: 0.114115 Aug 08 02:26:13.753860 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 75/157] 10.222.0.27 () {60 vars in 1316 bytes} [Mon Aug 8 02:26:13 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091/os-interface => generated 284 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 08 02:26:13.832690 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-54cd80d2-4558-4fba-8427-54c5fad4a47c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:13.833532 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-54cd80d2-4558-4fba-8427-54c5fad4a47c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: 95ae6b45-35d7-43a7-a021-91bb545cac4e] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:13.837664 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_api:SELECT=100 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:13.842130 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-54cd80d2-4558-4fba-8427-54c5fad4a47c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:13.842848 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-54cd80d2-4558-4fba-8427-54c5fad4a47c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:13.843496 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-54cd80d2-4558-4fba-8427-54c5fad4a47c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:13.844393 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-54cd80d2-4558-4fba-8427-54c5fad4a47c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:13.845098 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-54cd80d2-4558-4fba-8427-54c5fad4a47c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:13.845722 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-54cd80d2-4558-4fba-8427-54c5fad4a47c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:14.007679 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-54cd80d2-4558-4fba-8427-54c5fad4a47c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:14.018487 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-54cd80d2-4558-4fba-8427-54c5fad4a47c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e" status: 200 len: 1557 microversion: 2.1 time: 0.189305 Aug 08 02:26:14.019849 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 76/158] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:13 2022] GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e => generated 1557 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:26:14.042469 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-e304d0df-c64a-47f9-9a80-7472744f7414 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:14.043317 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-e304d0df-c64a-47f9-9a80-7472744f7414 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: 95ae6b45-35d7-43a7-a021-91bb545cac4e] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:14.052171 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e304d0df-c64a-47f9-9a80-7472744f7414 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:14.053220 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e304d0df-c64a-47f9-9a80-7472744f7414 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:14.053887 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e304d0df-c64a-47f9-9a80-7472744f7414 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:14.054943 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e304d0df-c64a-47f9-9a80-7472744f7414 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:14.055762 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e304d0df-c64a-47f9-9a80-7472744f7414 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:14.056387 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e304d0df-c64a-47f9-9a80-7472744f7414 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:14.149050 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-e304d0df-c64a-47f9-9a80-7472744f7414 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.111598 Aug 08 02:26:14.149644 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 77/159] 10.222.0.27 () {58 vars in 1275 bytes} [Mon Aug 8 02:26:14 2022] GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e/os-interface => generated 285 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:26:14.154650 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-9dcdde6a-e84c-4e8d-b2db-a5258a371c19 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:14.155433 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-9dcdde6a-e84c-4e8d-b2db-a5258a371c19 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:14.162271 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-9dcdde6a-e84c-4e8d-b2db-a5258a371c19 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:14.162975 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-9dcdde6a-e84c-4e8d-b2db-a5258a371c19 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:14.163605 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-9dcdde6a-e84c-4e8d-b2db-a5258a371c19 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:14.164315 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-9dcdde6a-e84c-4e8d-b2db-a5258a371c19 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:14.164789 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-9dcdde6a-e84c-4e8d-b2db-a5258a371c19 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:14.165512 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-9dcdde6a-e84c-4e8d-b2db-a5258a371c19 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:14.495672 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-9dcdde6a-e84c-4e8d-b2db-a5258a371c19 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:14.504795 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-9dcdde6a-e84c-4e8d-b2db-a5258a371c19 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1521 microversion: 2.1 time: 0.354004 Aug 08 02:26:14.510954 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 78/160] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:14 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1521 bytes in 356 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:14.873560 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-70294653-a015-4079-80c2-95a0819ed76f tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] 10.222.0.27 "POST /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.362364 Aug 08 02:26:14.874240 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 83/161] 10.222.0.27 () {60 vars in 1323 bytes} [Mon Aug 8 02:26:13 2022] POST /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae/os-volume_attachments => generated 194 bytes in 1364 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 08 02:26:15.374663 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [req-fc90977e-0a7e-483b-a729-e5dcebf45c5a req-f84c2f34-88d7-4566-9eb2-76b8739ad404 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f51a5900-5abc-4188-bbc4-33b0f0f7d091", "tag": "8d2a6aac-ff75-417c-b0b9-cbf799f88d22"}]} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:26:15.387455 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-fc90977e-0a7e-483b-a729-e5dcebf45c5a req-f84c2f34-88d7-4566-9eb2-76b8739ad404 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:15.389075 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-fc90977e-0a7e-483b-a729-e5dcebf45c5a req-f84c2f34-88d7-4566-9eb2-76b8739ad404 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:15.389729 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-fc90977e-0a7e-483b-a729-e5dcebf45c5a req-f84c2f34-88d7-4566-9eb2-76b8739ad404 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:15.404843 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.compute.server_external_events [req-fc90977e-0a7e-483b-a729-e5dcebf45c5a req-f84c2f34-88d7-4566-9eb2-76b8739ad404 service nova] Creating event network-changed:8d2a6aac-ff75-417c-b0b9-cbf799f88d22 for instance f51a5900-5abc-4188-bbc4-33b0f0f7d091 on hv-cinder-74014 Aug 08 02:26:15.411021 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [req-fc90977e-0a7e-483b-a729-e5dcebf45c5a req-f84c2f34-88d7-4566-9eb2-76b8739ad404 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041092 Aug 08 02:26:15.411829 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 79/162] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:26:15 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:26:15.527916 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-ddadb47f-d053-4ccd-b87f-8336b81258f3 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:15.529532 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-ddadb47f-d053-4ccd-b87f-8336b81258f3 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:15.538091 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ddadb47f-d053-4ccd-b87f-8336b81258f3 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:15.539295 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ddadb47f-d053-4ccd-b87f-8336b81258f3 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:15.539979 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ddadb47f-d053-4ccd-b87f-8336b81258f3 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:15.541652 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ddadb47f-d053-4ccd-b87f-8336b81258f3 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:15.542726 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ddadb47f-d053-4ccd-b87f-8336b81258f3 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:15.543916 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ddadb47f-d053-4ccd-b87f-8336b81258f3 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:15.841859 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-ddadb47f-d053-4ccd-b87f-8336b81258f3 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:15.857856 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-ddadb47f-d053-4ccd-b87f-8336b81258f3 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1521 microversion: 2.1 time: 0.333123 Aug 08 02:26:15.857856 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 84/163] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:15 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1521 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:26:16.076642 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [req-dce889e6-6367-4b24-9ee0-2a4c13721b43 req-bbe9b5cd-f5e9-4313-94e4-63457da49857 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "95ae6b45-35d7-43a7-a021-91bb545cac4e", "tag": "26e540df-eae8-4a69-873e-d4b1a210cfdf"}]} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:26:16.090548 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-dce889e6-6367-4b24-9ee0-2a4c13721b43 req-bbe9b5cd-f5e9-4313-94e4-63457da49857 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:16.092157 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-dce889e6-6367-4b24-9ee0-2a4c13721b43 req-bbe9b5cd-f5e9-4313-94e4-63457da49857 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:16.093083 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-dce889e6-6367-4b24-9ee0-2a4c13721b43 req-bbe9b5cd-f5e9-4313-94e4-63457da49857 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:16.113302 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.compute.server_external_events [req-dce889e6-6367-4b24-9ee0-2a4c13721b43 req-bbe9b5cd-f5e9-4313-94e4-63457da49857 service nova] Creating event network-changed:26e540df-eae8-4a69-873e-d4b1a210cfdf for instance 95ae6b45-35d7-43a7-a021-91bb545cac4e on hv-cinder-74014 Aug 08 02:26:16.120167 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [req-dce889e6-6367-4b24-9ee0-2a4c13721b43 req-bbe9b5cd-f5e9-4313-94e4-63457da49857 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.048012 Aug 08 02:26:16.121459 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 80/164] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:26:16 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:26:16.877527 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-949148a1-fd51-4be2-833c-79d91354ad50 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:16.878212 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-949148a1-fd51-4be2-833c-79d91354ad50 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:16.887634 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-949148a1-fd51-4be2-833c-79d91354ad50 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:16.888946 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-949148a1-fd51-4be2-833c-79d91354ad50 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:16.889619 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-949148a1-fd51-4be2-833c-79d91354ad50 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:16.890332 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-949148a1-fd51-4be2-833c-79d91354ad50 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:16.890794 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-949148a1-fd51-4be2-833c-79d91354ad50 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:16.896557 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-949148a1-fd51-4be2-833c-79d91354ad50 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:17.076993 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-949148a1-fd51-4be2-833c-79d91354ad50 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:17.086516 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-949148a1-fd51-4be2-833c-79d91354ad50 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1521 microversion: 2.1 time: 0.213972 Aug 08 02:26:17.087357 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 85/165] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:16 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1521 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:18.110502 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-996e057c-4186-4dae-8c18-0984e912fd39 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:18.111288 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-996e057c-4186-4dae-8c18-0984e912fd39 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:18.117957 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-996e057c-4186-4dae-8c18-0984e912fd39 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:18.118466 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-996e057c-4186-4dae-8c18-0984e912fd39 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:18.118902 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-996e057c-4186-4dae-8c18-0984e912fd39 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:18.119654 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-996e057c-4186-4dae-8c18-0984e912fd39 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:18.120133 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-996e057c-4186-4dae-8c18-0984e912fd39 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:18.120569 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-996e057c-4186-4dae-8c18-0984e912fd39 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:18.257898 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-996e057c-4186-4dae-8c18-0984e912fd39 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:18.266515 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-996e057c-4186-4dae-8c18-0984e912fd39 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1521 microversion: 2.1 time: 0.161096 Aug 08 02:26:18.267107 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 81/166] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:18 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1521 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:19.287355 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-3501adb6-edb3-4d9b-8857-50b02de8876b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:19.288897 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-3501adb6-edb3-4d9b-8857-50b02de8876b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:19.298023 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3501adb6-edb3-4d9b-8857-50b02de8876b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:19.299072 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3501adb6-edb3-4d9b-8857-50b02de8876b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:19.299675 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3501adb6-edb3-4d9b-8857-50b02de8876b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:19.300737 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3501adb6-edb3-4d9b-8857-50b02de8876b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:19.301349 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3501adb6-edb3-4d9b-8857-50b02de8876b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:19.301921 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3501adb6-edb3-4d9b-8857-50b02de8876b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:19.444139 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-3501adb6-edb3-4d9b-8857-50b02de8876b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:19.453013 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-3501adb6-edb3-4d9b-8857-50b02de8876b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1521 microversion: 2.1 time: 0.169491 Aug 08 02:26:19.453869 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 86/167] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:19 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1521 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:20.474876 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-1f89889b-f5bd-41ab-a7e8-0d9ed204e624 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:20.475480 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-1f89889b-f5bd-41ab-a7e8-0d9ed204e624 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:20.481567 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1f89889b-f5bd-41ab-a7e8-0d9ed204e624 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:20.482648 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1f89889b-f5bd-41ab-a7e8-0d9ed204e624 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:20.483300 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1f89889b-f5bd-41ab-a7e8-0d9ed204e624 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:20.483790 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1f89889b-f5bd-41ab-a7e8-0d9ed204e624 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:20.484237 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1f89889b-f5bd-41ab-a7e8-0d9ed204e624 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:20.484655 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1f89889b-f5bd-41ab-a7e8-0d9ed204e624 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:20.616289 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-1f89889b-f5bd-41ab-a7e8-0d9ed204e624 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:20.619902 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell1:SELECT=100 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:20.624489 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-1f89889b-f5bd-41ab-a7e8-0d9ed204e624 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1521 microversion: 2.1 time: 0.153613 Aug 08 02:26:20.625085 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 82/168] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:20 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1521 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:21.645314 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-1fdec575-2fc7-4279-b785-b11a8e0c1557 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:21.645924 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-1fdec575-2fc7-4279-b785-b11a8e0c1557 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:21.652438 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1fdec575-2fc7-4279-b785-b11a8e0c1557 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:21.652956 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1fdec575-2fc7-4279-b785-b11a8e0c1557 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:21.653394 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1fdec575-2fc7-4279-b785-b11a8e0c1557 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:21.654091 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1fdec575-2fc7-4279-b785-b11a8e0c1557 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:21.654545 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1fdec575-2fc7-4279-b785-b11a8e0c1557 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:21.654969 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1fdec575-2fc7-4279-b785-b11a8e0c1557 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:21.786963 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-1fdec575-2fc7-4279-b785-b11a8e0c1557 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:21.795221 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-1fdec575-2fc7-4279-b785-b11a8e0c1557 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1521 microversion: 2.1 time: 0.153886 Aug 08 02:26:21.795781 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 87/169] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:21 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1521 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:22.817003 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-42c150ef-6a2c-49f0-9a16-78f09c8a1a77 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:22.817639 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-42c150ef-6a2c-49f0-9a16-78f09c8a1a77 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:22.826120 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-42c150ef-6a2c-49f0-9a16-78f09c8a1a77 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:22.826613 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-42c150ef-6a2c-49f0-9a16-78f09c8a1a77 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:22.827047 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-42c150ef-6a2c-49f0-9a16-78f09c8a1a77 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:22.827736 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-42c150ef-6a2c-49f0-9a16-78f09c8a1a77 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:22.828971 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-42c150ef-6a2c-49f0-9a16-78f09c8a1a77 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:22.829403 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-42c150ef-6a2c-49f0-9a16-78f09c8a1a77 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:22.969853 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-42c150ef-6a2c-49f0-9a16-78f09c8a1a77 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:22.977352 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-42c150ef-6a2c-49f0-9a16-78f09c8a1a77 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1521 microversion: 2.1 time: 0.164408 Aug 08 02:26:22.977918 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 83/170] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:22 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1521 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:24.000167 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-99d583ef-5220-4cb6-9530-a88be6ca64fd tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:24.000817 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-99d583ef-5220-4cb6-9530-a88be6ca64fd tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:24.007275 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-99d583ef-5220-4cb6-9530-a88be6ca64fd tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:24.007759 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-99d583ef-5220-4cb6-9530-a88be6ca64fd tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:24.008195 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-99d583ef-5220-4cb6-9530-a88be6ca64fd tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:24.009158 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-99d583ef-5220-4cb6-9530-a88be6ca64fd tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:24.009714 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-99d583ef-5220-4cb6-9530-a88be6ca64fd tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:24.010047 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-99d583ef-5220-4cb6-9530-a88be6ca64fd tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:24.154656 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-99d583ef-5220-4cb6-9530-a88be6ca64fd tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:24.162225 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-99d583ef-5220-4cb6-9530-a88be6ca64fd tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1521 microversion: 2.1 time: 0.167055 Aug 08 02:26:24.162791 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 88/171] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:23 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1521 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:25.183155 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-41e3e667-c73c-4656-98ec-a878518005a8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:25.183760 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-41e3e667-c73c-4656-98ec-a878518005a8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:25.191406 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-41e3e667-c73c-4656-98ec-a878518005a8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:25.191881 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-41e3e667-c73c-4656-98ec-a878518005a8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:25.192310 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-41e3e667-c73c-4656-98ec-a878518005a8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:25.193027 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-41e3e667-c73c-4656-98ec-a878518005a8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:25.193475 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-41e3e667-c73c-4656-98ec-a878518005a8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:25.193971 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-41e3e667-c73c-4656-98ec-a878518005a8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:25.323496 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-41e3e667-c73c-4656-98ec-a878518005a8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:25.330938 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-41e3e667-c73c-4656-98ec-a878518005a8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1521 microversion: 2.1 time: 0.151957 Aug 08 02:26:25.331504 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 84/172] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:25 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1521 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:26.351211 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-9ab4915e-8f19-48e2-b5c2-08d5ceb9d185 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:26.351815 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-9ab4915e-8f19-48e2-b5c2-08d5ceb9d185 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:26.359492 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9ab4915e-8f19-48e2-b5c2-08d5ceb9d185 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:26.359968 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9ab4915e-8f19-48e2-b5c2-08d5ceb9d185 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:26.360447 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9ab4915e-8f19-48e2-b5c2-08d5ceb9d185 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:26.361153 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9ab4915e-8f19-48e2-b5c2-08d5ceb9d185 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:26.361601 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9ab4915e-8f19-48e2-b5c2-08d5ceb9d185 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:26.362047 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9ab4915e-8f19-48e2-b5c2-08d5ceb9d185 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:26.366103 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:26.495629 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-9ab4915e-8f19-48e2-b5c2-08d5ceb9d185 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:26.503338 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-9ab4915e-8f19-48e2-b5c2-08d5ceb9d185 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1521 microversion: 2.1 time: 0.155960 Aug 08 02:26:26.503906 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 89/173] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:26 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1521 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:27.523322 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-4ee8b7ed-194b-4bd9-8b2f-1e8e26d5590a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:27.523959 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-4ee8b7ed-194b-4bd9-8b2f-1e8e26d5590a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:27.530195 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4ee8b7ed-194b-4bd9-8b2f-1e8e26d5590a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:27.530679 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4ee8b7ed-194b-4bd9-8b2f-1e8e26d5590a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:27.531164 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4ee8b7ed-194b-4bd9-8b2f-1e8e26d5590a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:27.531876 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4ee8b7ed-194b-4bd9-8b2f-1e8e26d5590a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:27.532328 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4ee8b7ed-194b-4bd9-8b2f-1e8e26d5590a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:27.532776 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4ee8b7ed-194b-4bd9-8b2f-1e8e26d5590a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:27.662134 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-4ee8b7ed-194b-4bd9-8b2f-1e8e26d5590a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:27.670707 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-4ee8b7ed-194b-4bd9-8b2f-1e8e26d5590a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1521 microversion: 2.1 time: 0.151249 Aug 08 02:26:27.671276 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 85/174] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:27 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1521 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:28.692081 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-1bca2c4d-c15e-40d3-8569-459858b9a14d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:28.692701 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-1bca2c4d-c15e-40d3-8569-459858b9a14d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:28.699696 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1bca2c4d-c15e-40d3-8569-459858b9a14d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:28.700167 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1bca2c4d-c15e-40d3-8569-459858b9a14d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:28.701284 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1bca2c4d-c15e-40d3-8569-459858b9a14d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:28.701960 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1bca2c4d-c15e-40d3-8569-459858b9a14d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:28.702409 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1bca2c4d-c15e-40d3-8569-459858b9a14d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:28.702830 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1bca2c4d-c15e-40d3-8569-459858b9a14d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:28.830382 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-1bca2c4d-c15e-40d3-8569-459858b9a14d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:28.837818 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-1bca2c4d-c15e-40d3-8569-459858b9a14d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1521 microversion: 2.1 time: 0.149542 Aug 08 02:26:28.838445 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 90/175] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:28 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1521 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:26:29.859763 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-918748de-67d2-49d7-aecc-7361116f0cc2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:29.860328 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-918748de-67d2-49d7-aecc-7361116f0cc2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:29.866351 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-918748de-67d2-49d7-aecc-7361116f0cc2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:29.866824 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-918748de-67d2-49d7-aecc-7361116f0cc2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:29.867249 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-918748de-67d2-49d7-aecc-7361116f0cc2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:29.867955 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-918748de-67d2-49d7-aecc-7361116f0cc2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:29.868404 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-918748de-67d2-49d7-aecc-7361116f0cc2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:29.868853 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-918748de-67d2-49d7-aecc-7361116f0cc2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:30.004632 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-918748de-67d2-49d7-aecc-7361116f0cc2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:30.014536 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-918748de-67d2-49d7-aecc-7361116f0cc2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1521 microversion: 2.1 time: 0.159540 Aug 08 02:26:30.015121 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 86/176] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:29 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1521 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:31.054607 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-db85d7e6-f2b5-4185-9a76-e38c3f5f030c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:31.054607 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-db85d7e6-f2b5-4185-9a76-e38c3f5f030c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:31.064073 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-db85d7e6-f2b5-4185-9a76-e38c3f5f030c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:31.064570 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-db85d7e6-f2b5-4185-9a76-e38c3f5f030c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:31.065050 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-db85d7e6-f2b5-4185-9a76-e38c3f5f030c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:31.065871 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-db85d7e6-f2b5-4185-9a76-e38c3f5f030c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:31.066664 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-db85d7e6-f2b5-4185-9a76-e38c3f5f030c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:31.067086 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-db85d7e6-f2b5-4185-9a76-e38c3f5f030c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:31.209008 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-db85d7e6-f2b5-4185-9a76-e38c3f5f030c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:31.217019 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-db85d7e6-f2b5-4185-9a76-e38c3f5f030c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1521 microversion: 2.1 time: 0.174833 Aug 08 02:26:31.217599 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 91/177] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:31 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1521 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:32.191094 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-057465af-4f8f-4759-ae5c-7cd1c6d40a58 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:32.191612 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-057465af-4f8f-4759-ae5c-7cd1c6d40a58 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] [instance: 0fc3bfe1-b902-4a65-8c41-81e6f49112ae] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:32.197693 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-057465af-4f8f-4759-ae5c-7cd1c6d40a58 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:32.198130 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-057465af-4f8f-4759-ae5c-7cd1c6d40a58 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:32.198561 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-057465af-4f8f-4759-ae5c-7cd1c6d40a58 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:32.199248 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-057465af-4f8f-4759-ae5c-7cd1c6d40a58 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:32.199693 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-057465af-4f8f-4759-ae5c-7cd1c6d40a58 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:32.200125 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-057465af-4f8f-4759-ae5c-7cd1c6d40a58 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:32.226628 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-057465af-4f8f-4759-ae5c-7cd1c6d40a58 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] [instance: 0fc3bfe1-b902-4a65-8c41-81e6f49112ae] Going to try to terminate instance {{(pid=118351) delete /opt/stack/nova/nova/compute/api.py:2666}} Aug 08 02:26:32.239400 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-b1862c43-c2a0-4941-808b-4c7cb1cdb6ce tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:32.240030 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-b1862c43-c2a0-4941-808b-4c7cb1cdb6ce tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:32.248035 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b1862c43-c2a0-4941-808b-4c7cb1cdb6ce tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:32.248806 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b1862c43-c2a0-4941-808b-4c7cb1cdb6ce tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:32.249449 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b1862c43-c2a0-4941-808b-4c7cb1cdb6ce tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:32.253595 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b1862c43-c2a0-4941-808b-4c7cb1cdb6ce tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:32.253595 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b1862c43-c2a0-4941-808b-4c7cb1cdb6ce tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:32.253595 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b1862c43-c2a0-4941-808b-4c7cb1cdb6ce tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:32.394769 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-b1862c43-c2a0-4941-808b-4c7cb1cdb6ce tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:32.402404 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-b1862c43-c2a0-4941-808b-4c7cb1cdb6ce tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1521 microversion: 2.1 time: 0.166929 Aug 08 02:26:32.402976 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 92/178] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:32 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1521 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:26:32.665000 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-057465af-4f8f-4759-ae5c-7cd1c6d40a58 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] 10.222.0.27 "DELETE /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae" status: 204 len: 0 microversion: 2.1 time: 0.478015 Aug 08 02:26:32.665694 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 87/179] 10.222.0.27 () {58 vars in 1239 bytes} [Mon Aug 8 02:26:32 2022] DELETE /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae => generated 0 bytes in 481 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 08 02:26:32.674332 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-dd375986-7c13-4431-8c18-c2d56b3a7b9e tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:32.674941 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-dd375986-7c13-4431-8c18-c2d56b3a7b9e tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] [instance: 0fc3bfe1-b902-4a65-8c41-81e6f49112ae] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:32.680812 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dd375986-7c13-4431-8c18-c2d56b3a7b9e tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:32.681312 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dd375986-7c13-4431-8c18-c2d56b3a7b9e tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:32.681724 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dd375986-7c13-4431-8c18-c2d56b3a7b9e tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:32.682412 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dd375986-7c13-4431-8c18-c2d56b3a7b9e tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:32.682890 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dd375986-7c13-4431-8c18-c2d56b3a7b9e tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:32.683311 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dd375986-7c13-4431-8c18-c2d56b3a7b9e tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:32.823666 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-dd375986-7c13-4431-8c18-c2d56b3a7b9e tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2aeb52d51d3f44319bb4e114858222f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5515114a058944a49182063213de54d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:32.832902 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-dd375986-7c13-4431-8c18-c2d56b3a7b9e tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] 10.222.0.27 "GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae" status: 200 len: 1553 microversion: 2.1 time: 0.162384 Aug 08 02:26:32.833475 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 93/180] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:32 2022] GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae => generated 1553 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:33.424590 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-4b174386-bbcd-4a77-9253-24ee3ca0fcc8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:33.425268 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-4b174386-bbcd-4a77-9253-24ee3ca0fcc8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:33.431239 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4b174386-bbcd-4a77-9253-24ee3ca0fcc8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:33.431745 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4b174386-bbcd-4a77-9253-24ee3ca0fcc8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:33.432202 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4b174386-bbcd-4a77-9253-24ee3ca0fcc8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:33.432902 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4b174386-bbcd-4a77-9253-24ee3ca0fcc8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:33.433363 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4b174386-bbcd-4a77-9253-24ee3ca0fcc8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:33.433812 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4b174386-bbcd-4a77-9253-24ee3ca0fcc8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:33.566530 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-4b174386-bbcd-4a77-9253-24ee3ca0fcc8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:33.574545 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-4b174386-bbcd-4a77-9253-24ee3ca0fcc8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1521 microversion: 2.1 time: 0.154905 Aug 08 02:26:33.575126 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 88/181] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:33 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1521 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:33.854351 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-09a0e2dc-7eaf-4c12-ac4a-d085d1ccc1fe tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:33.854969 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-09a0e2dc-7eaf-4c12-ac4a-d085d1ccc1fe tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] [instance: 0fc3bfe1-b902-4a65-8c41-81e6f49112ae] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:33.861601 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-09a0e2dc-7eaf-4c12-ac4a-d085d1ccc1fe tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:33.862090 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-09a0e2dc-7eaf-4c12-ac4a-d085d1ccc1fe tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:33.862528 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-09a0e2dc-7eaf-4c12-ac4a-d085d1ccc1fe tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:33.863224 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-09a0e2dc-7eaf-4c12-ac4a-d085d1ccc1fe tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:33.863707 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-09a0e2dc-7eaf-4c12-ac4a-d085d1ccc1fe tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:33.864159 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-09a0e2dc-7eaf-4c12-ac4a-d085d1ccc1fe tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:34.006952 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-09a0e2dc-7eaf-4c12-ac4a-d085d1ccc1fe tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2aeb52d51d3f44319bb4e114858222f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5515114a058944a49182063213de54d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:34.015742 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-09a0e2dc-7eaf-4c12-ac4a-d085d1ccc1fe tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] 10.222.0.27 "GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae" status: 200 len: 1553 microversion: 2.1 time: 0.165283 Aug 08 02:26:34.016290 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 94/182] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:33 2022] GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae => generated 1553 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:34.597461 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-c1872b80-0e2f-4a0b-98ba-800fd162fc64 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:34.598072 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-c1872b80-0e2f-4a0b-98ba-800fd162fc64 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:34.604017 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c1872b80-0e2f-4a0b-98ba-800fd162fc64 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:34.604488 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c1872b80-0e2f-4a0b-98ba-800fd162fc64 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:34.604962 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c1872b80-0e2f-4a0b-98ba-800fd162fc64 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:34.605659 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c1872b80-0e2f-4a0b-98ba-800fd162fc64 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:34.606104 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c1872b80-0e2f-4a0b-98ba-800fd162fc64 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:34.606520 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c1872b80-0e2f-4a0b-98ba-800fd162fc64 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:34.738729 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-c1872b80-0e2f-4a0b-98ba-800fd162fc64 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:34.746308 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-c1872b80-0e2f-4a0b-98ba-800fd162fc64 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1521 microversion: 2.1 time: 0.152737 Aug 08 02:26:34.746973 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 89/183] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:34 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1521 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:35.037790 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-0ebbb846-9569-4ad3-b171-4e9d5d3dd311 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:35.038397 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-0ebbb846-9569-4ad3-b171-4e9d5d3dd311 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] [instance: 0fc3bfe1-b902-4a65-8c41-81e6f49112ae] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:35.052064 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0ebbb846-9569-4ad3-b171-4e9d5d3dd311 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:35.052064 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0ebbb846-9569-4ad3-b171-4e9d5d3dd311 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:35.052064 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0ebbb846-9569-4ad3-b171-4e9d5d3dd311 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:35.052064 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0ebbb846-9569-4ad3-b171-4e9d5d3dd311 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:35.052386 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0ebbb846-9569-4ad3-b171-4e9d5d3dd311 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:35.052386 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0ebbb846-9569-4ad3-b171-4e9d5d3dd311 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:35.184371 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-0ebbb846-9569-4ad3-b171-4e9d5d3dd311 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2aeb52d51d3f44319bb4e114858222f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5515114a058944a49182063213de54d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:35.193632 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-0ebbb846-9569-4ad3-b171-4e9d5d3dd311 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] 10.222.0.27 "GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae" status: 200 len: 1553 microversion: 2.1 time: 0.159495 Aug 08 02:26:35.194330 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 95/184] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:35 2022] GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae => generated 1553 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:35.768373 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-d7406a1a-dcb1-49f4-b82c-d0e7647eb32b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:35.769004 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-d7406a1a-dcb1-49f4-b82c-d0e7647eb32b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:35.776429 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d7406a1a-dcb1-49f4-b82c-d0e7647eb32b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:35.865840 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d7406a1a-dcb1-49f4-b82c-d0e7647eb32b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:35.865840 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d7406a1a-dcb1-49f4-b82c-d0e7647eb32b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:35.865840 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d7406a1a-dcb1-49f4-b82c-d0e7647eb32b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:35.865840 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d7406a1a-dcb1-49f4-b82c-d0e7647eb32b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:35.867190 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d7406a1a-dcb1-49f4-b82c-d0e7647eb32b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:35.923710 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-d7406a1a-dcb1-49f4-b82c-d0e7647eb32b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:35.932183 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-d7406a1a-dcb1-49f4-b82c-d0e7647eb32b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1521 microversion: 2.1 time: 0.167743 Aug 08 02:26:35.936865 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 90/185] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:35 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1521 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:36.215934 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-b66ce601-3f01-440f-9568-513eb9b27283 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:36.218542 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-b66ce601-3f01-440f-9568-513eb9b27283 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] [instance: 0fc3bfe1-b902-4a65-8c41-81e6f49112ae] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:36.246149 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b66ce601-3f01-440f-9568-513eb9b27283 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:36.253623 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [req-76ead42d-e020-4a86-b0a1-0b9b5f82cd70 req-302ed70e-c99e-4a25-b288-fba6317f2a35 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0fc3bfe1-b902-4a65-8c41-81e6f49112ae", "name": "network-vif-deleted", "tag": "5482fadc-3651-40bc-b0e6-82d5cf650c16"}]} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:26:36.258920 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b66ce601-3f01-440f-9568-513eb9b27283 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:36.261608 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b66ce601-3f01-440f-9568-513eb9b27283 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:36.264366 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-76ead42d-e020-4a86-b0a1-0b9b5f82cd70 req-302ed70e-c99e-4a25-b288-fba6317f2a35 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:36.265025 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-76ead42d-e020-4a86-b0a1-0b9b5f82cd70 req-302ed70e-c99e-4a25-b288-fba6317f2a35 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:36.265464 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-76ead42d-e020-4a86-b0a1-0b9b5f82cd70 req-302ed70e-c99e-4a25-b288-fba6317f2a35 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:36.265584 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b66ce601-3f01-440f-9568-513eb9b27283 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:36.268712 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b66ce601-3f01-440f-9568-513eb9b27283 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:36.273690 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b66ce601-3f01-440f-9568-513eb9b27283 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:36.281679 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.compute.server_external_events [req-76ead42d-e020-4a86-b0a1-0b9b5f82cd70 req-302ed70e-c99e-4a25-b288-fba6317f2a35 service nova] Creating event network-vif-deleted:5482fadc-3651-40bc-b0e6-82d5cf650c16 for instance 0fc3bfe1-b902-4a65-8c41-81e6f49112ae on hv-cinder-74014 Aug 08 02:26:36.293759 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [req-76ead42d-e020-4a86-b0a1-0b9b5f82cd70 req-302ed70e-c99e-4a25-b288-fba6317f2a35 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043809 Aug 08 02:26:36.295797 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 91/186] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:26:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 08 02:26:36.364048 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-b66ce601-3f01-440f-9568-513eb9b27283 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2aeb52d51d3f44319bb4e114858222f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5515114a058944a49182063213de54d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:36.375535 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-b66ce601-3f01-440f-9568-513eb9b27283 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] 10.222.0.27 "GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae" status: 200 len: 1511 microversion: 2.1 time: 0.164458 Aug 08 02:26:36.379033 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 96/187] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:36 2022] GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae => generated 1511 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:26:36.954548 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-f6c2edf0-d001-4f47-be22-e02d04ecee3c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:36.955274 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-f6c2edf0-d001-4f47-be22-e02d04ecee3c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:36.961589 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f6c2edf0-d001-4f47-be22-e02d04ecee3c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:36.962075 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f6c2edf0-d001-4f47-be22-e02d04ecee3c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:36.962519 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f6c2edf0-d001-4f47-be22-e02d04ecee3c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:36.963208 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f6c2edf0-d001-4f47-be22-e02d04ecee3c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:36.963665 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f6c2edf0-d001-4f47-be22-e02d04ecee3c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:36.964091 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f6c2edf0-d001-4f47-be22-e02d04ecee3c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:37.117094 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-f6c2edf0-d001-4f47-be22-e02d04ecee3c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:37.127010 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-f6c2edf0-d001-4f47-be22-e02d04ecee3c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1521 microversion: 2.1 time: 0.176356 Aug 08 02:26:37.127592 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 92/188] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:36 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1521 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:37.397767 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-c260ec8b-5797-464f-ae2e-a4889fb59986 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:37.398376 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-c260ec8b-5797-464f-ae2e-a4889fb59986 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] [instance: 0fc3bfe1-b902-4a65-8c41-81e6f49112ae] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:37.405686 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c260ec8b-5797-464f-ae2e-a4889fb59986 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:37.406221 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c260ec8b-5797-464f-ae2e-a4889fb59986 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:37.406916 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c260ec8b-5797-464f-ae2e-a4889fb59986 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:37.407601 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c260ec8b-5797-464f-ae2e-a4889fb59986 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:37.408049 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c260ec8b-5797-464f-ae2e-a4889fb59986 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:37.408469 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c260ec8b-5797-464f-ae2e-a4889fb59986 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:37.470007 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-c260ec8b-5797-464f-ae2e-a4889fb59986 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2aeb52d51d3f44319bb4e114858222f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5515114a058944a49182063213de54d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:37.477980 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-c260ec8b-5797-464f-ae2e-a4889fb59986 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] 10.222.0.27 "GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae" status: 200 len: 1345 microversion: 2.1 time: 0.084297 Aug 08 02:26:37.478552 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 97/189] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:37 2022] GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae => generated 1345 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:38.149106 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-23d5b4f6-b3a2-44fd-8d17-ae26ed4f80e6 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:38.149626 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-23d5b4f6-b3a2-44fd-8d17-ae26ed4f80e6 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:38.155899 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-23d5b4f6-b3a2-44fd-8d17-ae26ed4f80e6 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:38.156387 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-23d5b4f6-b3a2-44fd-8d17-ae26ed4f80e6 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:38.156847 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-23d5b4f6-b3a2-44fd-8d17-ae26ed4f80e6 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:38.157536 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-23d5b4f6-b3a2-44fd-8d17-ae26ed4f80e6 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:38.157988 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-23d5b4f6-b3a2-44fd-8d17-ae26ed4f80e6 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:38.158415 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-23d5b4f6-b3a2-44fd-8d17-ae26ed4f80e6 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:38.286390 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-23d5b4f6-b3a2-44fd-8d17-ae26ed4f80e6 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:38.294152 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-23d5b4f6-b3a2-44fd-8d17-ae26ed4f80e6 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1521 microversion: 2.1 time: 0.149269 Aug 08 02:26:38.295401 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 93/190] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:38 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1521 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:38.499892 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-acb16c26-9861-4de2-9ee0-2ac0b29940df tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:38.500494 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-acb16c26-9861-4de2-9ee0-2ac0b29940df tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] [instance: 0fc3bfe1-b902-4a65-8c41-81e6f49112ae] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:38.506984 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-acb16c26-9861-4de2-9ee0-2ac0b29940df tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:38.507455 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-acb16c26-9861-4de2-9ee0-2ac0b29940df tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:38.507886 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-acb16c26-9861-4de2-9ee0-2ac0b29940df tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:38.508577 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-acb16c26-9861-4de2-9ee0-2ac0b29940df tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:38.509049 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-acb16c26-9861-4de2-9ee0-2ac0b29940df tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:38.509468 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-acb16c26-9861-4de2-9ee0-2ac0b29940df tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:38.572346 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-acb16c26-9861-4de2-9ee0-2ac0b29940df tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2aeb52d51d3f44319bb4e114858222f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5515114a058944a49182063213de54d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:38.580475 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-acb16c26-9861-4de2-9ee0-2ac0b29940df tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] 10.222.0.27 "GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae" status: 200 len: 1345 microversion: 2.1 time: 0.085005 Aug 08 02:26:38.584085 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 98/191] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:38 2022] GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae => generated 1345 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:39.315770 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-8f5a3ef4-1f1c-465a-97d1-e3d1598fae57 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:39.316418 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-8f5a3ef4-1f1c-465a-97d1-e3d1598fae57 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:39.322470 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8f5a3ef4-1f1c-465a-97d1-e3d1598fae57 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:39.322962 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8f5a3ef4-1f1c-465a-97d1-e3d1598fae57 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:39.323390 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8f5a3ef4-1f1c-465a-97d1-e3d1598fae57 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:39.324071 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8f5a3ef4-1f1c-465a-97d1-e3d1598fae57 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:39.324522 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8f5a3ef4-1f1c-465a-97d1-e3d1598fae57 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:39.324968 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8f5a3ef4-1f1c-465a-97d1-e3d1598fae57 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:39.456766 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-8f5a3ef4-1f1c-465a-97d1-e3d1598fae57 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:39.464488 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-8f5a3ef4-1f1c-465a-97d1-e3d1598fae57 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1521 microversion: 2.1 time: 0.152943 Aug 08 02:26:39.465117 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 94/192] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:39 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1521 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:39.601733 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-a91b517e-f170-4bcb-bf8e-28ad8fea7826 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:39.602347 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-a91b517e-f170-4bcb-bf8e-28ad8fea7826 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] [instance: 0fc3bfe1-b902-4a65-8c41-81e6f49112ae] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:39.609497 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a91b517e-f170-4bcb-bf8e-28ad8fea7826 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:39.611146 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a91b517e-f170-4bcb-bf8e-28ad8fea7826 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:39.611146 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a91b517e-f170-4bcb-bf8e-28ad8fea7826 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:39.611146 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a91b517e-f170-4bcb-bf8e-28ad8fea7826 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:39.611542 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a91b517e-f170-4bcb-bf8e-28ad8fea7826 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:39.611969 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a91b517e-f170-4bcb-bf8e-28ad8fea7826 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:39.674467 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-a91b517e-f170-4bcb-bf8e-28ad8fea7826 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2aeb52d51d3f44319bb4e114858222f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5515114a058944a49182063213de54d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:39.686307 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-a91b517e-f170-4bcb-bf8e-28ad8fea7826 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] 10.222.0.27 "GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae" status: 200 len: 1345 microversion: 2.1 time: 0.085016 Aug 08 02:26:39.686307 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 99/193] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:39 2022] GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae => generated 1345 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:40.486367 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-dfd5f321-e7b8-4343-9fd2-7f89fbcccb30 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:40.487139 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-dfd5f321-e7b8-4343-9fd2-7f89fbcccb30 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:40.493523 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-dfd5f321-e7b8-4343-9fd2-7f89fbcccb30 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:40.494157 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-dfd5f321-e7b8-4343-9fd2-7f89fbcccb30 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:40.494655 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-dfd5f321-e7b8-4343-9fd2-7f89fbcccb30 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:40.495420 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-dfd5f321-e7b8-4343-9fd2-7f89fbcccb30 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:40.495939 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-dfd5f321-e7b8-4343-9fd2-7f89fbcccb30 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:40.496430 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-dfd5f321-e7b8-4343-9fd2-7f89fbcccb30 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:40.705098 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-90f16f7c-34c3-4ac1-b9a0-549bf53477b8 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:40.705820 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-90f16f7c-34c3-4ac1-b9a0-549bf53477b8 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] [instance: 0fc3bfe1-b902-4a65-8c41-81e6f49112ae] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:40.712421 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-90f16f7c-34c3-4ac1-b9a0-549bf53477b8 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:40.713036 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-90f16f7c-34c3-4ac1-b9a0-549bf53477b8 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:40.713541 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-90f16f7c-34c3-4ac1-b9a0-549bf53477b8 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:40.714387 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-90f16f7c-34c3-4ac1-b9a0-549bf53477b8 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:40.715060 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-90f16f7c-34c3-4ac1-b9a0-549bf53477b8 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:40.715780 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-90f16f7c-34c3-4ac1-b9a0-549bf53477b8 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:40.767364 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-90f16f7c-34c3-4ac1-b9a0-549bf53477b8 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:40.768107 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-90f16f7c-34c3-4ac1-b9a0-549bf53477b8 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:40.768727 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-90f16f7c-34c3-4ac1-b9a0-549bf53477b8 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:40.769580 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.objects.instance [None req-90f16f7c-34c3-4ac1-b9a0-549bf53477b8 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lazy-loading 'fault' on Instance uuid 0fc3bfe1-b902-4a65-8c41-81e6f49112ae {{(pid=118352) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 08 02:26:40.832796 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-90f16f7c-34c3-4ac1-b9a0-549bf53477b8 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2aeb52d51d3f44319bb4e114858222f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5515114a058944a49182063213de54d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:40.841785 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-90f16f7c-34c3-4ac1-b9a0-549bf53477b8 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] 10.222.0.27 "GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae" status: 200 len: 1350 microversion: 2.1 time: 0.141418 Aug 08 02:26:40.855354 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 100/194] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:40 2022] GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae => generated 1350 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:26:40.859497 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-dfd5f321-e7b8-4343-9fd2-7f89fbcccb30 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:40.867624 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-dfd5f321-e7b8-4343-9fd2-7f89fbcccb30 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1521 microversion: 2.1 time: 0.386290 Aug 08 02:26:40.868192 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 95/195] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:40 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1521 bytes in 387 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:41.864368 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-679a2c9a-e373-4395-8e77-a55521eeb17f tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:41.865002 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-679a2c9a-e373-4395-8e77-a55521eeb17f tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] [instance: 0fc3bfe1-b902-4a65-8c41-81e6f49112ae] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:41.867349 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_api:SELECT=49 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:41.875016 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-679a2c9a-e373-4395-8e77-a55521eeb17f tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:41.875495 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-679a2c9a-e373-4395-8e77-a55521eeb17f tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:41.875927 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-679a2c9a-e373-4395-8e77-a55521eeb17f tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:41.876598 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-679a2c9a-e373-4395-8e77-a55521eeb17f tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:41.877082 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-679a2c9a-e373-4395-8e77-a55521eeb17f tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:41.877502 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-679a2c9a-e373-4395-8e77-a55521eeb17f tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:41.889515 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-9ad3d951-94ac-4baa-bd26-2dc9cfa1f6fb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:41.890128 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-9ad3d951-94ac-4baa-bd26-2dc9cfa1f6fb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:41.891154 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.wsgi [None req-679a2c9a-e373-4395-8e77-a55521eeb17f tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] HTTP exception thrown: Instance 0fc3bfe1-b902-4a65-8c41-81e6f49112ae could not be found. Aug 08 02:26:41.895843 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-679a2c9a-e373-4395-8e77-a55521eeb17f tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Request method failure captured: Aug 08 02:26:41.895843 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: request: GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae HTTP/1.1 Aug 08 02:26:41.895843 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept: application/json Aug 08 02:26:41.895843 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept-Encoding: identity Aug 08 02:26:41.895843 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Connection: close Aug 08 02:26:41.895843 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Length: 0 Aug 08 02:26:41.895843 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Type: application/json Aug 08 02:26:41.895843 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Host: 10.222.0.27 Aug 08 02:26:41.895843 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Openstack-System-Scope: None Aug 08 02:26:41.895843 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: User-Agent: python-urllib3/1.26.11 Aug 08 02:26:41.895843 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Auth-Token: gAAAAABi8HQeqdzc0U1Bn6cyAWHiBXoKOvxh3Xu6_fHap2s2GaTlxMI_i9QKcYFiEvEvT83XytiKf2HYYNnKS8rjxuudP01Jg3cDtN27x_hWE66eW6ahiFhKo1CIexhHajo4m7KkUBA1YSKhbhNWDFr4R-e_lVTkDdxcTyPujtavy4Ags5nSwhc Aug 08 02:26:41.895843 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Id: None Aug 08 02:26:41.895843 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Name: None Aug 08 02:26:41.895843 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Identity-Status: Confirmed Aug 08 02:26:41.895843 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Is-Admin-Project: True Aug 08 02:26:41.895843 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Id: default Aug 08 02:26:41.895843 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Name: Default Aug 08 02:26:41.895843 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Id: 5515114a058944a49182063213de54d3 Aug 08 02:26:41.895843 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Name: tempest-DeleteServersTestJSON-2061947428 Aug 08 02:26:41.895843 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Role: reader,member Aug 08 02:26:41.895843 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Roles: reader,member Aug 08 02:26:41.897360 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/5515114a058944a49182063213de54d3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/5515114a058944a49182063213de54d3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_5515114a058944a49182063213de54d3", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/5515114a058944a49182063213de54d3"}]}] Aug 08 02:26:41.897360 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant: tempest-DeleteServersTestJSON-2061947428 Aug 08 02:26:41.897360 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Id: 5515114a058944a49182063213de54d3 Aug 08 02:26:41.897360 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Name: tempest-DeleteServersTestJSON-2061947428 Aug 08 02:26:41.897360 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User: tempest-DeleteServersTestJSON-2061947428-project Aug 08 02:26:41.897360 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Id: default Aug 08 02:26:41.897360 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Name: Default Aug 08 02:26:41.897360 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Id: 2aeb52d51d3f44319bb4e114858222f0 Aug 08 02:26:41.897360 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Name: tempest-DeleteServersTestJSON-2061947428-project Aug 08 02:26:41.897360 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: method: > Aug 08 02:26:41.897360 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: exception: Instance 0fc3bfe1-b902-4a65-8c41-81e6f49112ae could not be found. Aug 08 02:26:41.897360 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: action_args: {'id': '0fc3bfe1-b902-4a65-8c41-81e6f49112ae'} Aug 08 02:26:41.898092 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:26:41.898092 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:26:41.898092 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 08 02:26:41.898092 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 08 02:26:41.898092 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 08 02:26:41.898092 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 08 02:26:41.898092 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 08 02:26:41.898092 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 08 02:26:41.898092 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 08 02:26:41.898092 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 08 02:26:41.898092 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0fc3bfe1-b902-4a65-8c41-81e6f49112ae could not be found. Aug 08 02:26:41.898092 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:26:41.898092 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:26:41.898092 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:26:41.898092 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:26:41.898092 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:26:41.898092 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:26:41.898092 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:26:41.898092 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:26:41.898092 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:26:41.899294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:26:41.899294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 08 02:26:41.899294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 08 02:26:41.899294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 08 02:26:41.899294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 08 02:26:41.899294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 08 02:26:41.899294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:26:41.899294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0fc3bfe1-b902-4a65-8c41-81e6f49112ae could not be found. Aug 08 02:26:41.899294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:26:41.899294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:26:41.899294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:26:41.899294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:26:41.899294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:26:41.899294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:26:41.899294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:26:41.899294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:26:41.899294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0fc3bfe1-b902-4a65-8c41-81e6f49112ae could not be found. Aug 08 02:26:41.899294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:26:41.900378 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-9ad3d951-94ac-4baa-bd26-2dc9cfa1f6fb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:41.900378 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-9ad3d951-94ac-4baa-bd26-2dc9cfa1f6fb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:41.900522 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-679a2c9a-e373-4395-8e77-a55521eeb17f tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Returning 404 to user: Instance 0fc3bfe1-b902-4a65-8c41-81e6f49112ae could not be found. {{(pid=118352) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:26:41.900592 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-9ad3d951-94ac-4baa-bd26-2dc9cfa1f6fb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:41.900696 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-679a2c9a-e373-4395-8e77-a55521eeb17f tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] 10.222.0.27 "GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae" status: 404 len: 111 microversion: 2.1 time: 0.036874 Aug 08 02:26:41.900770 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-9ad3d951-94ac-4baa-bd26-2dc9cfa1f6fb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:41.900839 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 101/196] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:41 2022] GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 08 02:26:41.900905 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-9ad3d951-94ac-4baa-bd26-2dc9cfa1f6fb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:41.900905 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-9ad3d951-94ac-4baa-bd26-2dc9cfa1f6fb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:42.031533 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-9ad3d951-94ac-4baa-bd26-2dc9cfa1f6fb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:42.039353 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-9ad3d951-94ac-4baa-bd26-2dc9cfa1f6fb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1521 microversion: 2.1 time: 0.154070 Aug 08 02:26:42.039932 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 96/197] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:41 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1521 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:26:42.084363 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-0062feb3-7192-4024-9a9f-ef22b6d20d9e tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:42.086746 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-0062feb3-7192-4024-9a9f-ef22b6d20d9e tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] [instance: 0fc3bfe1-b902-4a65-8c41-81e6f49112ae] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:42.093461 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0062feb3-7192-4024-9a9f-ef22b6d20d9e tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:42.100160 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0062feb3-7192-4024-9a9f-ef22b6d20d9e tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:42.100160 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0062feb3-7192-4024-9a9f-ef22b6d20d9e tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:42.100160 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0062feb3-7192-4024-9a9f-ef22b6d20d9e tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:42.100160 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0062feb3-7192-4024-9a9f-ef22b6d20d9e tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:42.100496 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0062feb3-7192-4024-9a9f-ef22b6d20d9e tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:42.109534 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.wsgi [None req-0062feb3-7192-4024-9a9f-ef22b6d20d9e tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] HTTP exception thrown: Instance 0fc3bfe1-b902-4a65-8c41-81e6f49112ae could not be found. Aug 08 02:26:42.111073 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-0062feb3-7192-4024-9a9f-ef22b6d20d9e tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Request method failure captured: Aug 08 02:26:42.111073 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: request: GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae/os-volume_attachments HTTP/1.1 Aug 08 02:26:42.111073 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept: application/json Aug 08 02:26:42.111073 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept-Encoding: identity Aug 08 02:26:42.111073 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Connection: close Aug 08 02:26:42.111073 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Length: 0 Aug 08 02:26:42.111073 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Type: application/json Aug 08 02:26:42.111073 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Host: 10.222.0.27 Aug 08 02:26:42.111073 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Openstack-System-Scope: None Aug 08 02:26:42.111073 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: User-Agent: python-urllib3/1.26.11 Aug 08 02:26:42.111073 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Auth-Token: gAAAAABi8HQeqdzc0U1Bn6cyAWHiBXoKOvxh3Xu6_fHap2s2GaTlxMI_i9QKcYFiEvEvT83XytiKf2HYYNnKS8rjxuudP01Jg3cDtN27x_hWE66eW6ahiFhKo1CIexhHajo4m7KkUBA1YSKhbhNWDFr4R-e_lVTkDdxcTyPujtavy4Ags5nSwhc Aug 08 02:26:42.111073 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Id: None Aug 08 02:26:42.111073 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Name: None Aug 08 02:26:42.111073 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Identity-Status: Confirmed Aug 08 02:26:42.111073 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Is-Admin-Project: True Aug 08 02:26:42.111073 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Id: default Aug 08 02:26:42.111073 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Name: Default Aug 08 02:26:42.111073 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Id: 5515114a058944a49182063213de54d3 Aug 08 02:26:42.111073 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Name: tempest-DeleteServersTestJSON-2061947428 Aug 08 02:26:42.111073 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Role: reader,member Aug 08 02:26:42.111073 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Roles: reader,member Aug 08 02:26:42.112453 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/5515114a058944a49182063213de54d3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/5515114a058944a49182063213de54d3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_5515114a058944a49182063213de54d3", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/5515114a058944a49182063213de54d3"}]}] Aug 08 02:26:42.112453 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant: tempest-DeleteServersTestJSON-2061947428 Aug 08 02:26:42.112453 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Id: 5515114a058944a49182063213de54d3 Aug 08 02:26:42.112453 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Name: tempest-DeleteServersTestJSON-2061947428 Aug 08 02:26:42.112453 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User: tempest-DeleteServersTestJSON-2061947428-project Aug 08 02:26:42.112453 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Id: default Aug 08 02:26:42.112453 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Name: Default Aug 08 02:26:42.112453 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Id: 2aeb52d51d3f44319bb4e114858222f0 Aug 08 02:26:42.112453 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Name: tempest-DeleteServersTestJSON-2061947428-project Aug 08 02:26:42.112453 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: method: > Aug 08 02:26:42.112453 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: exception: Instance 0fc3bfe1-b902-4a65-8c41-81e6f49112ae could not be found. Aug 08 02:26:42.112453 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: action_args: {'server_id': '0fc3bfe1-b902-4a65-8c41-81e6f49112ae'} Aug 08 02:26:42.113229 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:26:42.113229 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:26:42.113229 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 08 02:26:42.113229 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 08 02:26:42.113229 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 08 02:26:42.113229 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 08 02:26:42.113229 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 08 02:26:42.113229 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 08 02:26:42.113229 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 08 02:26:42.113229 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 08 02:26:42.113229 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0fc3bfe1-b902-4a65-8c41-81e6f49112ae could not be found. Aug 08 02:26:42.113229 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:26:42.113229 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:26:42.113229 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:26:42.113229 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:26:42.113229 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:26:42.113229 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:26:42.113229 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:26:42.113229 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:26:42.113229 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:26:42.114413 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:26:42.114413 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Aug 08 02:26:42.114413 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 08 02:26:42.114413 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Aug 08 02:26:42.114413 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 08 02:26:42.114413 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 303, in index Aug 08 02:26:42.114413 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, server_id) Aug 08 02:26:42.114413 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 08 02:26:42.114413 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:26:42.114413 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0fc3bfe1-b902-4a65-8c41-81e6f49112ae could not be found. Aug 08 02:26:42.114413 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:26:42.114413 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:26:42.114413 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:26:42.114413 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:26:42.114413 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:26:42.114413 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:26:42.114413 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:26:42.114413 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:26:42.114413 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0fc3bfe1-b902-4a65-8c41-81e6f49112ae could not be found. Aug 08 02:26:42.114413 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:26:42.115786 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-0062feb3-7192-4024-9a9f-ef22b6d20d9e tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Returning 404 to user: Instance 0fc3bfe1-b902-4a65-8c41-81e6f49112ae could not be found. {{(pid=118352) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:26:42.115786 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-0062feb3-7192-4024-9a9f-ef22b6d20d9e tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] 10.222.0.27 "GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae/os-volume_attachments" status: 404 len: 111 microversion: 2.1 time: 0.032688 Aug 08 02:26:42.115786 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 102/198] 10.222.0.27 () {58 vars in 1302 bytes} [Mon Aug 8 02:26:42 2022] GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae/os-volume_attachments => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 08 02:26:43.059383 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-b7adc6d4-b291-4b42-85cb-6f2e956dcdfb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:43.060046 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-b7adc6d4-b291-4b42-85cb-6f2e956dcdfb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:43.066318 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b7adc6d4-b291-4b42-85cb-6f2e956dcdfb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:43.066800 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b7adc6d4-b291-4b42-85cb-6f2e956dcdfb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:43.067227 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b7adc6d4-b291-4b42-85cb-6f2e956dcdfb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:43.067905 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b7adc6d4-b291-4b42-85cb-6f2e956dcdfb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:43.068351 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b7adc6d4-b291-4b42-85cb-6f2e956dcdfb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:43.068795 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b7adc6d4-b291-4b42-85cb-6f2e956dcdfb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:43.203368 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-b7adc6d4-b291-4b42-85cb-6f2e956dcdfb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:43.210950 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-b7adc6d4-b291-4b42-85cb-6f2e956dcdfb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1521 microversion: 2.1 time: 0.155275 Aug 08 02:26:43.211564 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 97/199] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:43 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1521 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:44.231126 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-8e2f4fc9-e194-40df-9885-f23dfcf787b8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:44.231772 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-8e2f4fc9-e194-40df-9885-f23dfcf787b8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:44.238706 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8e2f4fc9-e194-40df-9885-f23dfcf787b8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:44.239190 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8e2f4fc9-e194-40df-9885-f23dfcf787b8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:44.239626 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8e2f4fc9-e194-40df-9885-f23dfcf787b8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:44.240308 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8e2f4fc9-e194-40df-9885-f23dfcf787b8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:44.240793 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8e2f4fc9-e194-40df-9885-f23dfcf787b8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:44.241219 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8e2f4fc9-e194-40df-9885-f23dfcf787b8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:44.366420 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-8e2f4fc9-e194-40df-9885-f23dfcf787b8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:44.374918 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-8e2f4fc9-e194-40df-9885-f23dfcf787b8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1521 microversion: 2.1 time: 0.147477 Aug 08 02:26:44.377382 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 103/200] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:44 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1521 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:45.395723 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-e87e6471-7c43-42c7-894a-20605dee1c8d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:45.396954 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-e87e6471-7c43-42c7-894a-20605dee1c8d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:45.399182 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_api:SELECT=48,nova_api:UPDATE=1 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:45.404556 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e87e6471-7c43-42c7-894a-20605dee1c8d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:45.405072 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e87e6471-7c43-42c7-894a-20605dee1c8d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:45.406115 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e87e6471-7c43-42c7-894a-20605dee1c8d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:45.406895 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e87e6471-7c43-42c7-894a-20605dee1c8d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:45.407353 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e87e6471-7c43-42c7-894a-20605dee1c8d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:45.407778 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e87e6471-7c43-42c7-894a-20605dee1c8d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:45.426085 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:45.542530 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-e87e6471-7c43-42c7-894a-20605dee1c8d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:45.550114 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-e87e6471-7c43-42c7-894a-20605dee1c8d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1521 microversion: 2.1 time: 0.158193 Aug 08 02:26:45.550678 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 98/201] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:45 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1521 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:45.628448 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-cef4cf85-497c-47f0-a64a-652f126924f5 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:45.628448 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-cef4cf85-497c-47f0-a64a-652f126924f5 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] [instance: 0fc3bfe1-b902-4a65-8c41-81e6f49112ae] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:45.633322 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cef4cf85-497c-47f0-a64a-652f126924f5 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:45.634002 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cef4cf85-497c-47f0-a64a-652f126924f5 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:45.634442 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cef4cf85-497c-47f0-a64a-652f126924f5 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:45.635145 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cef4cf85-497c-47f0-a64a-652f126924f5 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:45.635660 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cef4cf85-497c-47f0-a64a-652f126924f5 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:45.636032 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cef4cf85-497c-47f0-a64a-652f126924f5 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:45.650901 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.wsgi [None req-cef4cf85-497c-47f0-a64a-652f126924f5 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] HTTP exception thrown: Instance 0fc3bfe1-b902-4a65-8c41-81e6f49112ae could not be found. Aug 08 02:26:45.654172 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-cef4cf85-497c-47f0-a64a-652f126924f5 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Request method failure captured: Aug 08 02:26:45.654172 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: request: DELETE /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae HTTP/1.1 Aug 08 02:26:45.654172 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept: application/json Aug 08 02:26:45.654172 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept-Encoding: identity Aug 08 02:26:45.654172 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Connection: close Aug 08 02:26:45.654172 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Length: 0 Aug 08 02:26:45.654172 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Type: application/json Aug 08 02:26:45.654172 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Host: 10.222.0.27 Aug 08 02:26:45.654172 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Openstack-System-Scope: None Aug 08 02:26:45.654172 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: User-Agent: python-urllib3/1.26.11 Aug 08 02:26:45.654172 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Auth-Token: gAAAAABi8HQeqdzc0U1Bn6cyAWHiBXoKOvxh3Xu6_fHap2s2GaTlxMI_i9QKcYFiEvEvT83XytiKf2HYYNnKS8rjxuudP01Jg3cDtN27x_hWE66eW6ahiFhKo1CIexhHajo4m7KkUBA1YSKhbhNWDFr4R-e_lVTkDdxcTyPujtavy4Ags5nSwhc Aug 08 02:26:45.654172 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Id: None Aug 08 02:26:45.654172 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Name: None Aug 08 02:26:45.654172 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Identity-Status: Confirmed Aug 08 02:26:45.654172 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Is-Admin-Project: True Aug 08 02:26:45.654172 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Id: default Aug 08 02:26:45.654172 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Name: Default Aug 08 02:26:45.654172 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Id: 5515114a058944a49182063213de54d3 Aug 08 02:26:45.654172 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Name: tempest-DeleteServersTestJSON-2061947428 Aug 08 02:26:45.654172 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Role: reader,member Aug 08 02:26:45.654172 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Roles: reader,member Aug 08 02:26:45.659217 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/5515114a058944a49182063213de54d3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/5515114a058944a49182063213de54d3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_5515114a058944a49182063213de54d3", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/5515114a058944a49182063213de54d3"}]}] Aug 08 02:26:45.659217 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant: tempest-DeleteServersTestJSON-2061947428 Aug 08 02:26:45.659217 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Id: 5515114a058944a49182063213de54d3 Aug 08 02:26:45.659217 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Name: tempest-DeleteServersTestJSON-2061947428 Aug 08 02:26:45.659217 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User: tempest-DeleteServersTestJSON-2061947428-project Aug 08 02:26:45.659217 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Id: default Aug 08 02:26:45.659217 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Name: Default Aug 08 02:26:45.659217 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Id: 2aeb52d51d3f44319bb4e114858222f0 Aug 08 02:26:45.659217 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Name: tempest-DeleteServersTestJSON-2061947428-project Aug 08 02:26:45.659217 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: method: > Aug 08 02:26:45.659217 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: exception: Instance 0fc3bfe1-b902-4a65-8c41-81e6f49112ae could not be found. Aug 08 02:26:45.659217 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: action_args: {'id': '0fc3bfe1-b902-4a65-8c41-81e6f49112ae'} Aug 08 02:26:45.659894 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:26:45.659894 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:26:45.659894 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 08 02:26:45.659894 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 08 02:26:45.659894 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 08 02:26:45.659894 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 08 02:26:45.659894 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 08 02:26:45.659894 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 08 02:26:45.659894 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 08 02:26:45.659894 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 08 02:26:45.659894 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0fc3bfe1-b902-4a65-8c41-81e6f49112ae could not be found. Aug 08 02:26:45.659894 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:26:45.659894 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:26:45.659894 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:26:45.659894 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:26:45.659894 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:26:45.659894 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:26:45.659894 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:26:45.659894 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:26:45.659894 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:26:45.661013 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:26:45.661013 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 08 02:26:45.661013 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 08 02:26:45.661013 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 08 02:26:45.661013 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 08 02:26:45.661013 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 08 02:26:45.661013 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 08 02:26:45.661013 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 08 02:26:45.661013 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:26:45.661013 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0fc3bfe1-b902-4a65-8c41-81e6f49112ae could not be found. Aug 08 02:26:45.661013 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:26:45.661013 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:26:45.661013 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:26:45.661013 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:26:45.661013 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:26:45.661013 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:26:45.661013 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:26:45.661013 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:26:45.661013 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0fc3bfe1-b902-4a65-8c41-81e6f49112ae could not be found. Aug 08 02:26:45.661013 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:26:45.662127 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-cef4cf85-497c-47f0-a64a-652f126924f5 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Returning 404 to user: Instance 0fc3bfe1-b902-4a65-8c41-81e6f49112ae could not be found. {{(pid=118352) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:26:45.662127 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-cef4cf85-497c-47f0-a64a-652f126924f5 tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] 10.222.0.27 "DELETE /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae" status: 404 len: 111 microversion: 2.1 time: 0.033041 Aug 08 02:26:45.662127 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 104/202] 10.222.0.27 () {58 vars in 1239 bytes} [Mon Aug 8 02:26:45 2022] DELETE /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 08 02:26:45.663949 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-5d9a1978-d381-479d-b335-50f3457afadd tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:45.664575 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-5d9a1978-d381-479d-b335-50f3457afadd tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] [instance: 0fc3bfe1-b902-4a65-8c41-81e6f49112ae] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:45.671232 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5d9a1978-d381-479d-b335-50f3457afadd tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:45.671726 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5d9a1978-d381-479d-b335-50f3457afadd tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:45.672159 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5d9a1978-d381-479d-b335-50f3457afadd tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:45.673269 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5d9a1978-d381-479d-b335-50f3457afadd tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:45.673878 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5d9a1978-d381-479d-b335-50f3457afadd tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:45.674489 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5d9a1978-d381-479d-b335-50f3457afadd tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:45.689838 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.wsgi [None req-5d9a1978-d381-479d-b335-50f3457afadd tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] HTTP exception thrown: Instance 0fc3bfe1-b902-4a65-8c41-81e6f49112ae could not be found. Aug 08 02:26:45.694618 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-5d9a1978-d381-479d-b335-50f3457afadd tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Request method failure captured: Aug 08 02:26:45.694618 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: request: GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae HTTP/1.1 Aug 08 02:26:45.694618 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Accept: application/json Aug 08 02:26:45.694618 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Accept-Encoding: identity Aug 08 02:26:45.694618 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Connection: close Aug 08 02:26:45.694618 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Content-Length: 0 Aug 08 02:26:45.694618 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Content-Type: application/json Aug 08 02:26:45.694618 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Host: 10.222.0.27 Aug 08 02:26:45.694618 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Openstack-System-Scope: None Aug 08 02:26:45.694618 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: User-Agent: python-urllib3/1.26.11 Aug 08 02:26:45.694618 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Auth-Token: gAAAAABi8HQeqdzc0U1Bn6cyAWHiBXoKOvxh3Xu6_fHap2s2GaTlxMI_i9QKcYFiEvEvT83XytiKf2HYYNnKS8rjxuudP01Jg3cDtN27x_hWE66eW6ahiFhKo1CIexhHajo4m7KkUBA1YSKhbhNWDFr4R-e_lVTkDdxcTyPujtavy4Ags5nSwhc Aug 08 02:26:45.694618 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Domain-Id: None Aug 08 02:26:45.694618 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Domain-Name: None Aug 08 02:26:45.694618 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Identity-Status: Confirmed Aug 08 02:26:45.694618 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Is-Admin-Project: True Aug 08 02:26:45.694618 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Domain-Id: default Aug 08 02:26:45.694618 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Domain-Name: Default Aug 08 02:26:45.694618 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Id: 5515114a058944a49182063213de54d3 Aug 08 02:26:45.694618 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Name: tempest-DeleteServersTestJSON-2061947428 Aug 08 02:26:45.694618 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Role: reader,member Aug 08 02:26:45.694618 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Roles: reader,member Aug 08 02:26:45.695838 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/5515114a058944a49182063213de54d3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/5515114a058944a49182063213de54d3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_5515114a058944a49182063213de54d3", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/5515114a058944a49182063213de54d3"}]}] Aug 08 02:26:45.695838 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant: tempest-DeleteServersTestJSON-2061947428 Aug 08 02:26:45.695838 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant-Id: 5515114a058944a49182063213de54d3 Aug 08 02:26:45.695838 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant-Name: tempest-DeleteServersTestJSON-2061947428 Aug 08 02:26:45.695838 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User: tempest-DeleteServersTestJSON-2061947428-project Aug 08 02:26:45.695838 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Domain-Id: default Aug 08 02:26:45.695838 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Domain-Name: Default Aug 08 02:26:45.695838 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Id: 2aeb52d51d3f44319bb4e114858222f0 Aug 08 02:26:45.695838 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Name: tempest-DeleteServersTestJSON-2061947428-project Aug 08 02:26:45.695838 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: method: > Aug 08 02:26:45.695838 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: exception: Instance 0fc3bfe1-b902-4a65-8c41-81e6f49112ae could not be found. Aug 08 02:26:45.695838 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: action_args: {'id': '0fc3bfe1-b902-4a65-8c41-81e6f49112ae'} Aug 08 02:26:45.696569 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:26:45.696569 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:26:45.696569 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 08 02:26:45.696569 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 08 02:26:45.696569 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 08 02:26:45.696569 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 08 02:26:45.696569 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 08 02:26:45.696569 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 08 02:26:45.696569 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 08 02:26:45.696569 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 08 02:26:45.696569 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0fc3bfe1-b902-4a65-8c41-81e6f49112ae could not be found. Aug 08 02:26:45.696569 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:26:45.696569 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:26:45.696569 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:26:45.696569 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:26:45.696569 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:26:45.696569 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:26:45.696569 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:26:45.696569 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:26:45.696569 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:26:45.700991 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:26:45.700991 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 08 02:26:45.700991 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 08 02:26:45.700991 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 08 02:26:45.700991 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 08 02:26:45.700991 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 08 02:26:45.700991 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:26:45.700991 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0fc3bfe1-b902-4a65-8c41-81e6f49112ae could not be found. Aug 08 02:26:45.700991 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:26:45.700991 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:26:45.700991 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:26:45.700991 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:26:45.700991 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:26:45.700991 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:26:45.700991 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:26:45.700991 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:26:45.700991 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0fc3bfe1-b902-4a65-8c41-81e6f49112ae could not be found. Aug 08 02:26:45.700991 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:26:45.702871 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-5d9a1978-d381-479d-b335-50f3457afadd tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] Returning 404 to user: Instance 0fc3bfe1-b902-4a65-8c41-81e6f49112ae could not be found. {{(pid=118351) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:26:45.702871 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-5d9a1978-d381-479d-b335-50f3457afadd tempest-DeleteServersTestJSON-2061947428 tempest-DeleteServersTestJSON-2061947428-project] 10.222.0.27 "GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae" status: 404 len: 111 microversion: 2.1 time: 0.036655 Aug 08 02:26:45.702871 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 99/203] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:45 2022] GET /compute/v2.1/servers/0fc3bfe1-b902-4a65-8c41-81e6f49112ae => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 08 02:26:46.571057 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-b35a574a-f575-462e-b05d-0ae34887c008 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:46.571667 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-b35a574a-f575-462e-b05d-0ae34887c008 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:46.588766 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b35a574a-f575-462e-b05d-0ae34887c008 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:46.588766 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b35a574a-f575-462e-b05d-0ae34887c008 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:46.588766 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b35a574a-f575-462e-b05d-0ae34887c008 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:46.588766 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b35a574a-f575-462e-b05d-0ae34887c008 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:46.589862 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b35a574a-f575-462e-b05d-0ae34887c008 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:46.589862 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b35a574a-f575-462e-b05d-0ae34887c008 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:46.759049 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-b35a574a-f575-462e-b05d-0ae34887c008 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:46.768136 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-b35a574a-f575-462e-b05d-0ae34887c008 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1521 microversion: 2.1 time: 0.200926 Aug 08 02:26:46.769096 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 105/204] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:46 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1521 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:47.789546 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-c9483c01-8c6e-444f-9edf-dec8726df576 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:47.790776 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-c9483c01-8c6e-444f-9edf-dec8726df576 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:47.797887 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c9483c01-8c6e-444f-9edf-dec8726df576 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:47.798597 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c9483c01-8c6e-444f-9edf-dec8726df576 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:47.799214 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c9483c01-8c6e-444f-9edf-dec8726df576 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:47.800081 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c9483c01-8c6e-444f-9edf-dec8726df576 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:47.800702 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c9483c01-8c6e-444f-9edf-dec8726df576 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:47.801636 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c9483c01-8c6e-444f-9edf-dec8726df576 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:48.021944 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-c9483c01-8c6e-444f-9edf-dec8726df576 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:48.031626 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-c9483c01-8c6e-444f-9edf-dec8726df576 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1521 microversion: 2.1 time: 0.244777 Aug 08 02:26:48.032533 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 100/205] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:47 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1521 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:49.053846 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-8e266f20-06bd-46cd-94e0-c5b558135ca8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:49.054499 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-8e266f20-06bd-46cd-94e0-c5b558135ca8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:49.061149 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8e266f20-06bd-46cd-94e0-c5b558135ca8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:49.062723 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8e266f20-06bd-46cd-94e0-c5b558135ca8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:49.063303 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8e266f20-06bd-46cd-94e0-c5b558135ca8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:49.064059 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8e266f20-06bd-46cd-94e0-c5b558135ca8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:49.064517 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8e266f20-06bd-46cd-94e0-c5b558135ca8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:49.065013 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8e266f20-06bd-46cd-94e0-c5b558135ca8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:49.249948 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-8e266f20-06bd-46cd-94e0-c5b558135ca8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:49.262032 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-8e266f20-06bd-46cd-94e0-c5b558135ca8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1521 microversion: 2.1 time: 0.212395 Aug 08 02:26:49.278024 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 106/206] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:49 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1521 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:50.286272 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-52a6f6bf-a519-4022-b6bc-a422a33cf963 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:50.287570 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-52a6f6bf-a519-4022-b6bc-a422a33cf963 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:50.294347 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-52a6f6bf-a519-4022-b6bc-a422a33cf963 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:50.295169 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-52a6f6bf-a519-4022-b6bc-a422a33cf963 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:50.295956 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-52a6f6bf-a519-4022-b6bc-a422a33cf963 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:50.297548 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-52a6f6bf-a519-4022-b6bc-a422a33cf963 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:50.298308 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-52a6f6bf-a519-4022-b6bc-a422a33cf963 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:50.299054 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-52a6f6bf-a519-4022-b6bc-a422a33cf963 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:50.453229 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-52a6f6bf-a519-4022-b6bc-a422a33cf963 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:50.460705 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-52a6f6bf-a519-4022-b6bc-a422a33cf963 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1521 microversion: 2.1 time: 0.178267 Aug 08 02:26:50.461440 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 101/207] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:50 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1521 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:51.485880 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-04f187c2-e317-4047-b080-e8d290c3f26f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:51.485880 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-04f187c2-e317-4047-b080-e8d290c3f26f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:51.491664 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-04f187c2-e317-4047-b080-e8d290c3f26f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:51.492367 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-04f187c2-e317-4047-b080-e8d290c3f26f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:51.493068 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-04f187c2-e317-4047-b080-e8d290c3f26f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:51.494002 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-04f187c2-e317-4047-b080-e8d290c3f26f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:51.494528 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-04f187c2-e317-4047-b080-e8d290c3f26f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:51.495028 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-04f187c2-e317-4047-b080-e8d290c3f26f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:51.536631 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell1:SELECT=100 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:51.675562 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-04f187c2-e317-4047-b080-e8d290c3f26f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:51.684084 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-04f187c2-e317-4047-b080-e8d290c3f26f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1521 microversion: 2.1 time: 0.205674 Aug 08 02:26:51.684871 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 107/208] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:51 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1521 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:52.708226 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-04dafd48-6193-4ae0-b726-c2148e3521f8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:52.709395 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-04dafd48-6193-4ae0-b726-c2148e3521f8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:52.715683 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-04dafd48-6193-4ae0-b726-c2148e3521f8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:52.717201 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-04dafd48-6193-4ae0-b726-c2148e3521f8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:52.717804 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-04dafd48-6193-4ae0-b726-c2148e3521f8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:52.718843 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-04dafd48-6193-4ae0-b726-c2148e3521f8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:52.719929 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-04dafd48-6193-4ae0-b726-c2148e3521f8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:52.720504 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-04dafd48-6193-4ae0-b726-c2148e3521f8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:52.887283 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-04dafd48-6193-4ae0-b726-c2148e3521f8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:52.895576 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-04dafd48-6193-4ae0-b726-c2148e3521f8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1521 microversion: 2.1 time: 0.191383 Aug 08 02:26:52.896145 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 102/209] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:52 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1521 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:53.917134 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-d96aac10-ca65-4d79-83a4-a161d1ed0c31 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:53.917822 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-d96aac10-ca65-4d79-83a4-a161d1ed0c31 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:53.926188 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d96aac10-ca65-4d79-83a4-a161d1ed0c31 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:53.926752 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d96aac10-ca65-4d79-83a4-a161d1ed0c31 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:53.927357 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d96aac10-ca65-4d79-83a4-a161d1ed0c31 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:53.928210 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d96aac10-ca65-4d79-83a4-a161d1ed0c31 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:53.928659 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d96aac10-ca65-4d79-83a4-a161d1ed0c31 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:53.929100 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d96aac10-ca65-4d79-83a4-a161d1ed0c31 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:54.075219 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-d96aac10-ca65-4d79-83a4-a161d1ed0c31 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:54.080785 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-d96aac10-ca65-4d79-83a4-a161d1ed0c31 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1521 microversion: 2.1 time: 0.168203 Aug 08 02:26:54.094141 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 108/210] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:53 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1521 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:55.102942 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-aac47358-59f4-4c83-a83a-14f0ff651e1d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:55.103851 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-aac47358-59f4-4c83-a83a-14f0ff651e1d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:55.113440 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-aac47358-59f4-4c83-a83a-14f0ff651e1d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:55.113440 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-aac47358-59f4-4c83-a83a-14f0ff651e1d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:55.113440 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-aac47358-59f4-4c83-a83a-14f0ff651e1d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:55.113440 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-aac47358-59f4-4c83-a83a-14f0ff651e1d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:55.114197 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-aac47358-59f4-4c83-a83a-14f0ff651e1d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:55.114197 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-aac47358-59f4-4c83-a83a-14f0ff651e1d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:55.244990 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-aac47358-59f4-4c83-a83a-14f0ff651e1d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:55.256828 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-aac47358-59f4-4c83-a83a-14f0ff651e1d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1501 microversion: 2.1 time: 0.158174 Aug 08 02:26:55.257396 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 103/211] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:55 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1501 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:55.277031 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-1c7371bb-bef6-4d05-b990-3f0481a63ab6 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:55.277644 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-1c7371bb-bef6-4d05-b990-3f0481a63ab6 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: ed75f58f-9135-4d97-b24a-2f8fcdab53ce] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:55.284865 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1c7371bb-bef6-4d05-b990-3f0481a63ab6 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:55.285357 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1c7371bb-bef6-4d05-b990-3f0481a63ab6 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:55.285791 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1c7371bb-bef6-4d05-b990-3f0481a63ab6 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:55.286479 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1c7371bb-bef6-4d05-b990-3f0481a63ab6 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:55.286932 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1c7371bb-bef6-4d05-b990-3f0481a63ab6 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:55.287355 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1c7371bb-bef6-4d05-b990-3f0481a63ab6 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:55.431090 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-1c7371bb-bef6-4d05-b990-3f0481a63ab6 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:55.439425 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-1c7371bb-bef6-4d05-b990-3f0481a63ab6 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce" status: 200 len: 1516 microversion: 2.1 time: 0.166142 Aug 08 02:26:55.440033 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 109/212] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:55 2022] GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce => generated 1516 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:26:57.628414 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-f0183e88-2063-4086-9a6c-452dbdde4295 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:26:57.631298 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-f0183e88-2063-4086-9a6c-452dbdde4295 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: ed75f58f-9135-4d97-b24a-2f8fcdab53ce] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:57.638869 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f0183e88-2063-4086-9a6c-452dbdde4295 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:57.639658 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f0183e88-2063-4086-9a6c-452dbdde4295 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:57.640394 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f0183e88-2063-4086-9a6c-452dbdde4295 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:57.641334 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f0183e88-2063-4086-9a6c-452dbdde4295 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:57.642332 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f0183e88-2063-4086-9a6c-452dbdde4295 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:57.643258 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f0183e88-2063-4086-9a6c-452dbdde4295 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:57.667153 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.objects.instance [None req-f0183e88-2063-4086-9a6c-452dbdde4295 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lazy-loading 'flavor' on Instance uuid ed75f58f-9135-4d97-b24a-2f8fcdab53ce {{(pid=118351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 08 02:26:58.022612 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-f0183e88-2063-4086-9a6c-452dbdde4295 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "POST /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce/action" status: 200 len: 32 microversion: 2.1 time: 0.398774 Aug 08 02:26:58.023218 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 104/213] 10.222.0.27 () {60 vars in 1278 bytes} [Mon Aug 8 02:26:57 2022] POST /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce/action => generated 32 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Aug 08 02:26:58.035756 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-473acd55-1631-4852-ae53-ccd36e24f6f8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:58.035756 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-473acd55-1631-4852-ae53-ccd36e24f6f8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: ed75f58f-9135-4d97-b24a-2f8fcdab53ce] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:58.041887 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-473acd55-1631-4852-ae53-ccd36e24f6f8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:58.042366 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-473acd55-1631-4852-ae53-ccd36e24f6f8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:58.042804 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-473acd55-1631-4852-ae53-ccd36e24f6f8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:58.043487 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-473acd55-1631-4852-ae53-ccd36e24f6f8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:58.043934 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-473acd55-1631-4852-ae53-ccd36e24f6f8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:58.044356 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-473acd55-1631-4852-ae53-ccd36e24f6f8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:58.181298 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-473acd55-1631-4852-ae53-ccd36e24f6f8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:58.196405 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-473acd55-1631-4852-ae53-ccd36e24f6f8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce" status: 200 len: 1522 microversion: 2.1 time: 0.160819 Aug 08 02:26:58.196405 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 110/214] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:58 2022] GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce => generated 1522 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:26:59.212992 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-5e918b4a-e749-4ace-89d4-167447ae212e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:26:59.213690 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-5e918b4a-e749-4ace-89d4-167447ae212e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: ed75f58f-9135-4d97-b24a-2f8fcdab53ce] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:26:59.222357 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5e918b4a-e749-4ace-89d4-167447ae212e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:59.222357 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5e918b4a-e749-4ace-89d4-167447ae212e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:59.222357 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5e918b4a-e749-4ace-89d4-167447ae212e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:59.222357 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5e918b4a-e749-4ace-89d4-167447ae212e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:26:59.222707 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5e918b4a-e749-4ace-89d4-167447ae212e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:26:59.223764 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5e918b4a-e749-4ace-89d4-167447ae212e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:26:59.361904 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-5e918b4a-e749-4ace-89d4-167447ae212e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:26:59.369475 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-5e918b4a-e749-4ace-89d4-167447ae212e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce" status: 200 len: 1522 microversion: 2.1 time: 0.160862 Aug 08 02:26:59.370043 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 105/215] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:26:59 2022] GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce => generated 1522 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:00.391588 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-9aa0730b-8a23-4e63-9a26-606e82783ec6 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:00.392144 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-9aa0730b-8a23-4e63-9a26-606e82783ec6 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: ed75f58f-9135-4d97-b24a-2f8fcdab53ce] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:00.399026 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9aa0730b-8a23-4e63-9a26-606e82783ec6 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:00.399594 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9aa0730b-8a23-4e63-9a26-606e82783ec6 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:00.400034 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9aa0730b-8a23-4e63-9a26-606e82783ec6 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:00.400801 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9aa0730b-8a23-4e63-9a26-606e82783ec6 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:00.401260 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9aa0730b-8a23-4e63-9a26-606e82783ec6 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:00.401691 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9aa0730b-8a23-4e63-9a26-606e82783ec6 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:00.535666 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-9aa0730b-8a23-4e63-9a26-606e82783ec6 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:00.543324 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-9aa0730b-8a23-4e63-9a26-606e82783ec6 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce" status: 200 len: 1522 microversion: 2.1 time: 0.157036 Aug 08 02:27:00.543929 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 111/216] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:00 2022] GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce => generated 1522 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:01.565079 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-84c59e9e-0eae-40da-a5bd-11b1c906b3dc tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:01.565690 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-84c59e9e-0eae-40da-a5bd-11b1c906b3dc tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: ed75f58f-9135-4d97-b24a-2f8fcdab53ce] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:01.571644 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-84c59e9e-0eae-40da-a5bd-11b1c906b3dc tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:01.572120 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-84c59e9e-0eae-40da-a5bd-11b1c906b3dc tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:01.572550 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-84c59e9e-0eae-40da-a5bd-11b1c906b3dc tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:01.573253 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-84c59e9e-0eae-40da-a5bd-11b1c906b3dc tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:01.573706 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-84c59e9e-0eae-40da-a5bd-11b1c906b3dc tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:01.574137 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-84c59e9e-0eae-40da-a5bd-11b1c906b3dc tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:01.707737 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-84c59e9e-0eae-40da-a5bd-11b1c906b3dc tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:01.715297 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-84c59e9e-0eae-40da-a5bd-11b1c906b3dc tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce" status: 200 len: 1522 microversion: 2.1 time: 0.154392 Aug 08 02:27:01.715863 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 106/217] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:01 2022] GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce => generated 1522 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:02.737456 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-d39c376b-a0ac-41cc-aa18-cd7445fe4cfa tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:02.738055 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-d39c376b-a0ac-41cc-aa18-cd7445fe4cfa tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: ed75f58f-9135-4d97-b24a-2f8fcdab53ce] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:02.744420 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d39c376b-a0ac-41cc-aa18-cd7445fe4cfa tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:02.744930 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d39c376b-a0ac-41cc-aa18-cd7445fe4cfa tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:02.745361 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d39c376b-a0ac-41cc-aa18-cd7445fe4cfa tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:02.746052 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d39c376b-a0ac-41cc-aa18-cd7445fe4cfa tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:02.746499 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d39c376b-a0ac-41cc-aa18-cd7445fe4cfa tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:02.747086 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d39c376b-a0ac-41cc-aa18-cd7445fe4cfa tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:02.879716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-d39c376b-a0ac-41cc-aa18-cd7445fe4cfa tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:02.887551 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-d39c376b-a0ac-41cc-aa18-cd7445fe4cfa tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce" status: 200 len: 1522 microversion: 2.1 time: 0.154881 Aug 08 02:27:02.888115 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 112/218] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:02 2022] GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce => generated 1522 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:03.909278 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-e94a518c-9396-46ac-96a6-5a6c0377b9b1 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:03.910082 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-e94a518c-9396-46ac-96a6-5a6c0377b9b1 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: ed75f58f-9135-4d97-b24a-2f8fcdab53ce] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:03.915994 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e94a518c-9396-46ac-96a6-5a6c0377b9b1 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:03.916477 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e94a518c-9396-46ac-96a6-5a6c0377b9b1 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:03.916932 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e94a518c-9396-46ac-96a6-5a6c0377b9b1 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:03.917675 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e94a518c-9396-46ac-96a6-5a6c0377b9b1 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:03.918131 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e94a518c-9396-46ac-96a6-5a6c0377b9b1 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:03.918554 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e94a518c-9396-46ac-96a6-5a6c0377b9b1 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:04.069867 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-e94a518c-9396-46ac-96a6-5a6c0377b9b1 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:04.076436 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-e94a518c-9396-46ac-96a6-5a6c0377b9b1 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce" status: 200 len: 1522 microversion: 2.1 time: 0.171625 Aug 08 02:27:04.077234 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 107/219] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:03 2022] GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce => generated 1522 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:05.099660 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-c78865c9-2330-4caa-93a4-2b2c95d21abf tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:05.101416 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-c78865c9-2330-4caa-93a4-2b2c95d21abf tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: ed75f58f-9135-4d97-b24a-2f8fcdab53ce] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:05.107032 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c78865c9-2330-4caa-93a4-2b2c95d21abf tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:05.107541 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c78865c9-2330-4caa-93a4-2b2c95d21abf tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:05.108001 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c78865c9-2330-4caa-93a4-2b2c95d21abf tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:05.108826 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c78865c9-2330-4caa-93a4-2b2c95d21abf tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:05.109287 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c78865c9-2330-4caa-93a4-2b2c95d21abf tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:05.109720 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c78865c9-2330-4caa-93a4-2b2c95d21abf tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:05.245905 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-c78865c9-2330-4caa-93a4-2b2c95d21abf tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:05.253336 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-c78865c9-2330-4caa-93a4-2b2c95d21abf tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce" status: 200 len: 1522 microversion: 2.1 time: 0.157551 Aug 08 02:27:05.253902 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 113/220] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:05 2022] GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce => generated 1522 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:06.273495 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-9a6b361f-e8a1-4a7b-a5aa-10d5f8e30a87 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:06.274164 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-9a6b361f-e8a1-4a7b-a5aa-10d5f8e30a87 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: ed75f58f-9135-4d97-b24a-2f8fcdab53ce] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:06.280124 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-9a6b361f-e8a1-4a7b-a5aa-10d5f8e30a87 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:06.280738 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-9a6b361f-e8a1-4a7b-a5aa-10d5f8e30a87 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:06.281273 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-9a6b361f-e8a1-4a7b-a5aa-10d5f8e30a87 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:06.281960 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-9a6b361f-e8a1-4a7b-a5aa-10d5f8e30a87 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:06.282413 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-9a6b361f-e8a1-4a7b-a5aa-10d5f8e30a87 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:06.282834 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-9a6b361f-e8a1-4a7b-a5aa-10d5f8e30a87 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:06.408259 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-9a6b361f-e8a1-4a7b-a5aa-10d5f8e30a87 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:06.415757 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-9a6b361f-e8a1-4a7b-a5aa-10d5f8e30a87 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce" status: 200 len: 1522 microversion: 2.1 time: 0.146082 Aug 08 02:27:06.416325 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 108/221] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:06 2022] GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce => generated 1522 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:07.289664 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-386cf162-2e00-4f03-84a1-34d1361258c7 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:07.290238 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-386cf162-2e00-4f03-84a1-34d1361258c7 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:07.297199 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-386cf162-2e00-4f03-84a1-34d1361258c7 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:07.297677 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-386cf162-2e00-4f03-84a1-34d1361258c7 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:07.298167 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-386cf162-2e00-4f03-84a1-34d1361258c7 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:07.298854 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-386cf162-2e00-4f03-84a1-34d1361258c7 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:07.299312 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-386cf162-2e00-4f03-84a1-34d1361258c7 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:07.299730 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-386cf162-2e00-4f03-84a1-34d1361258c7 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:07.322783 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-386cf162-2e00-4f03-84a1-34d1361258c7 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a/ips" status: 200 len: 147 microversion: 2.1 time: 0.037015 Aug 08 02:27:07.323340 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 114/222] 10.222.0.27 () {58 vars in 1248 bytes} [Mon Aug 8 02:27:07 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a/ips => generated 147 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:27:07.438223 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-c23e6fe7-30d2-40ae-a110-daceb20ab0a5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:07.438874 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-c23e6fe7-30d2-40ae-a110-daceb20ab0a5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: ed75f58f-9135-4d97-b24a-2f8fcdab53ce] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:07.445060 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c23e6fe7-30d2-40ae-a110-daceb20ab0a5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:07.445546 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c23e6fe7-30d2-40ae-a110-daceb20ab0a5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:07.445981 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c23e6fe7-30d2-40ae-a110-daceb20ab0a5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:07.446668 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c23e6fe7-30d2-40ae-a110-daceb20ab0a5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:07.447121 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c23e6fe7-30d2-40ae-a110-daceb20ab0a5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:07.447569 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c23e6fe7-30d2-40ae-a110-daceb20ab0a5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:07.579633 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-c23e6fe7-30d2-40ae-a110-daceb20ab0a5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:07.587490 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-c23e6fe7-30d2-40ae-a110-daceb20ab0a5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce" status: 200 len: 1522 microversion: 2.1 time: 0.154132 Aug 08 02:27:07.588067 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 109/223] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:07 2022] GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce => generated 1522 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:08.609101 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-a39448d9-d274-4dc8-83b2-3684c4c1f37d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:08.609710 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-a39448d9-d274-4dc8-83b2-3684c4c1f37d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: ed75f58f-9135-4d97-b24a-2f8fcdab53ce] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:08.616097 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a39448d9-d274-4dc8-83b2-3684c4c1f37d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:08.616574 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a39448d9-d274-4dc8-83b2-3684c4c1f37d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:08.617050 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a39448d9-d274-4dc8-83b2-3684c4c1f37d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:08.617731 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a39448d9-d274-4dc8-83b2-3684c4c1f37d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:08.618183 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a39448d9-d274-4dc8-83b2-3684c4c1f37d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:08.618605 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a39448d9-d274-4dc8-83b2-3684c4c1f37d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:08.748317 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-a39448d9-d274-4dc8-83b2-3684c4c1f37d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:08.756976 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-a39448d9-d274-4dc8-83b2-3684c4c1f37d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce" status: 200 len: 1522 microversion: 2.1 time: 0.151889 Aug 08 02:27:08.757600 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 115/224] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:08 2022] GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce => generated 1522 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:09.778384 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-b51bebef-5827-46c2-98c6-b1f55acf18cc tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:09.778905 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-b51bebef-5827-46c2-98c6-b1f55acf18cc tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: ed75f58f-9135-4d97-b24a-2f8fcdab53ce] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:09.786000 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b51bebef-5827-46c2-98c6-b1f55acf18cc tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:09.786504 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b51bebef-5827-46c2-98c6-b1f55acf18cc tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:09.786968 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b51bebef-5827-46c2-98c6-b1f55acf18cc tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:09.787645 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b51bebef-5827-46c2-98c6-b1f55acf18cc tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:09.788095 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b51bebef-5827-46c2-98c6-b1f55acf18cc tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:09.788522 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b51bebef-5827-46c2-98c6-b1f55acf18cc tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:09.921895 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-b51bebef-5827-46c2-98c6-b1f55acf18cc tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:09.929917 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-b51bebef-5827-46c2-98c6-b1f55acf18cc tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce" status: 200 len: 1522 microversion: 2.1 time: 0.155403 Aug 08 02:27:09.930473 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 110/225] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:09 2022] GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce => generated 1522 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:10.955008 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-f1edc5a0-482e-45b2-991c-9168fc0baf4c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:10.955800 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-f1edc5a0-482e-45b2-991c-9168fc0baf4c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: ed75f58f-9135-4d97-b24a-2f8fcdab53ce] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:10.963148 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f1edc5a0-482e-45b2-991c-9168fc0baf4c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:10.963737 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f1edc5a0-482e-45b2-991c-9168fc0baf4c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:10.964277 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f1edc5a0-482e-45b2-991c-9168fc0baf4c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:10.965110 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f1edc5a0-482e-45b2-991c-9168fc0baf4c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:10.965658 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f1edc5a0-482e-45b2-991c-9168fc0baf4c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:10.966167 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f1edc5a0-482e-45b2-991c-9168fc0baf4c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:11.102324 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-f1edc5a0-482e-45b2-991c-9168fc0baf4c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:11.109881 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-f1edc5a0-482e-45b2-991c-9168fc0baf4c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce" status: 200 len: 1522 microversion: 2.1 time: 0.162080 Aug 08 02:27:11.110446 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 116/226] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:10 2022] GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce => generated 1522 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:12.131256 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-705bbfb8-dc09-4313-84a4-0ccce1aaa62f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:12.131859 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-705bbfb8-dc09-4313-84a4-0ccce1aaa62f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: ed75f58f-9135-4d97-b24a-2f8fcdab53ce] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:12.138243 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-705bbfb8-dc09-4313-84a4-0ccce1aaa62f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:12.138721 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-705bbfb8-dc09-4313-84a4-0ccce1aaa62f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:12.139151 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-705bbfb8-dc09-4313-84a4-0ccce1aaa62f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:12.139821 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-705bbfb8-dc09-4313-84a4-0ccce1aaa62f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:12.140270 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-705bbfb8-dc09-4313-84a4-0ccce1aaa62f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:12.140703 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-705bbfb8-dc09-4313-84a4-0ccce1aaa62f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:12.267058 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-705bbfb8-dc09-4313-84a4-0ccce1aaa62f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:12.274433 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-705bbfb8-dc09-4313-84a4-0ccce1aaa62f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce" status: 200 len: 1522 microversion: 2.1 time: 0.147022 Aug 08 02:27:12.274995 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 111/227] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:12 2022] GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce => generated 1522 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:12.657310 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-05723200-7f51-46b8-b269-1930c98d0d56 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:12.657861 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-05723200-7f51-46b8-b269-1930c98d0d56 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: 95ae6b45-35d7-43a7-a021-91bb545cac4e] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:12.660055 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_api:SELECT=32 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:12.664749 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-05723200-7f51-46b8-b269-1930c98d0d56 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:12.665227 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-05723200-7f51-46b8-b269-1930c98d0d56 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:12.665657 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-05723200-7f51-46b8-b269-1930c98d0d56 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:12.666335 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-05723200-7f51-46b8-b269-1930c98d0d56 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:12.666783 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-05723200-7f51-46b8-b269-1930c98d0d56 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:12.667198 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-05723200-7f51-46b8-b269-1930c98d0d56 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:12.692899 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-05723200-7f51-46b8-b269-1930c98d0d56 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e/ips" status: 200 len: 151 microversion: 2.1 time: 0.039409 Aug 08 02:27:12.693840 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 117/228] 10.222.0.27 () {58 vars in 1248 bytes} [Mon Aug 8 02:27:12 2022] GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e/ips => generated 151 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:27:13.294525 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-8e3a2b59-1bbd-4ab2-8edc-7869bd8d9cb9 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:13.295139 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-8e3a2b59-1bbd-4ab2-8edc-7869bd8d9cb9 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: ed75f58f-9135-4d97-b24a-2f8fcdab53ce] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:13.301538 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8e3a2b59-1bbd-4ab2-8edc-7869bd8d9cb9 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:13.302022 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8e3a2b59-1bbd-4ab2-8edc-7869bd8d9cb9 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:13.302526 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8e3a2b59-1bbd-4ab2-8edc-7869bd8d9cb9 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:13.303213 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8e3a2b59-1bbd-4ab2-8edc-7869bd8d9cb9 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:13.303723 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8e3a2b59-1bbd-4ab2-8edc-7869bd8d9cb9 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:13.304146 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8e3a2b59-1bbd-4ab2-8edc-7869bd8d9cb9 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:13.446434 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-8e3a2b59-1bbd-4ab2-8edc-7869bd8d9cb9 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:13.454834 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-8e3a2b59-1bbd-4ab2-8edc-7869bd8d9cb9 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce" status: 200 len: 1522 microversion: 2.1 time: 0.164316 Aug 08 02:27:13.455406 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 112/229] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:13 2022] GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce => generated 1522 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:13.712116 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-a35a2e49-00cd-4ea8-a11f-8425b9fe723c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "a4ca4896-7117-4711-b088-f45bc051b0d4"}} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:27:13.714498 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-a35a2e49-00cd-4ea8-a11f-8425b9fe723c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:13.721399 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a35a2e49-00cd-4ea8-a11f-8425b9fe723c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:13.721880 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a35a2e49-00cd-4ea8-a11f-8425b9fe723c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:13.722319 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a35a2e49-00cd-4ea8-a11f-8425b9fe723c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:13.723008 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a35a2e49-00cd-4ea8-a11f-8425b9fe723c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:13.723458 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a35a2e49-00cd-4ea8-a11f-8425b9fe723c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:13.723882 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a35a2e49-00cd-4ea8-a11f-8425b9fe723c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:14.476062 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-2209a736-14a2-42a8-a217-90c46c3c6e93 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:14.476599 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-2209a736-14a2-42a8-a217-90c46c3c6e93 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: ed75f58f-9135-4d97-b24a-2f8fcdab53ce] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:14.482601 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2209a736-14a2-42a8-a217-90c46c3c6e93 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:14.483108 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2209a736-14a2-42a8-a217-90c46c3c6e93 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:14.484131 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2209a736-14a2-42a8-a217-90c46c3c6e93 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:14.485051 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2209a736-14a2-42a8-a217-90c46c3c6e93 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:14.485509 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2209a736-14a2-42a8-a217-90c46c3c6e93 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:14.490358 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2209a736-14a2-42a8-a217-90c46c3c6e93 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:14.622593 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-2209a736-14a2-42a8-a217-90c46c3c6e93 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:14.626554 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:14.631188 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-2209a736-14a2-42a8-a217-90c46c3c6e93 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce" status: 200 len: 1522 microversion: 2.1 time: 0.159027 Aug 08 02:27:14.631849 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 113/230] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:14 2022] GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce => generated 1522 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:27:15.652483 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-09becc76-9e42-4348-ac9c-526c39d04cd0 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:15.657043 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-09becc76-9e42-4348-ac9c-526c39d04cd0 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: ed75f58f-9135-4d97-b24a-2f8fcdab53ce] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:15.660076 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-09becc76-9e42-4348-ac9c-526c39d04cd0 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:15.660639 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-09becc76-9e42-4348-ac9c-526c39d04cd0 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:15.661129 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-09becc76-9e42-4348-ac9c-526c39d04cd0 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:15.661829 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-09becc76-9e42-4348-ac9c-526c39d04cd0 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:15.662286 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-09becc76-9e42-4348-ac9c-526c39d04cd0 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:15.662767 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-09becc76-9e42-4348-ac9c-526c39d04cd0 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:15.793831 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-09becc76-9e42-4348-ac9c-526c39d04cd0 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:15.802759 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-09becc76-9e42-4348-ac9c-526c39d04cd0 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce" status: 200 len: 1522 microversion: 2.1 time: 0.154492 Aug 08 02:27:15.803488 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 114/231] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:15 2022] GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce => generated 1522 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:27:16.824712 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-5e5d6563-867e-48c8-b13f-c0b6cbf16d46 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:16.825360 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-5e5d6563-867e-48c8-b13f-c0b6cbf16d46 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: ed75f58f-9135-4d97-b24a-2f8fcdab53ce] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:16.827553 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_api:SELECT=34 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:16.832403 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5e5d6563-867e-48c8-b13f-c0b6cbf16d46 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:16.832901 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5e5d6563-867e-48c8-b13f-c0b6cbf16d46 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:16.833330 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5e5d6563-867e-48c8-b13f-c0b6cbf16d46 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:16.834043 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5e5d6563-867e-48c8-b13f-c0b6cbf16d46 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:16.834651 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5e5d6563-867e-48c8-b13f-c0b6cbf16d46 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:16.835074 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5e5d6563-867e-48c8-b13f-c0b6cbf16d46 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:17.000395 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-5e5d6563-867e-48c8-b13f-c0b6cbf16d46 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:17.007884 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-5e5d6563-867e-48c8-b13f-c0b6cbf16d46 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce" status: 200 len: 1522 microversion: 2.1 time: 0.187519 Aug 08 02:27:17.008461 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 115/232] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:16 2022] GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce => generated 1522 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:27:17.156117 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-a35a2e49-00cd-4ea8-a11f-8425b9fe723c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "POST /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 3.446774 Aug 08 02:27:17.168453 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 118/233] 10.222.0.27 () {60 vars in 1323 bytes} [Mon Aug 8 02:27:13 2022] POST /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a/os-volume_attachments => generated 194 bytes in 3449 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:27:17.978663 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-b19456a5-c39d-46bd-8d4b-5f5ec2bbee2b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "bacd7028-1966-45e9-bc6e-a6156c1d3960"}} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:27:17.983034 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-b19456a5-c39d-46bd-8d4b-5f5ec2bbee2b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: 95ae6b45-35d7-43a7-a021-91bb545cac4e] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:17.992867 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b19456a5-c39d-46bd-8d4b-5f5ec2bbee2b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:17.992867 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b19456a5-c39d-46bd-8d4b-5f5ec2bbee2b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:17.992867 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b19456a5-c39d-46bd-8d4b-5f5ec2bbee2b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:17.992867 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b19456a5-c39d-46bd-8d4b-5f5ec2bbee2b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:17.993141 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b19456a5-c39d-46bd-8d4b-5f5ec2bbee2b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:17.993141 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b19456a5-c39d-46bd-8d4b-5f5ec2bbee2b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:18.034064 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-3c9574f0-719a-4020-9537-455c4e7af78c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:18.035017 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-3c9574f0-719a-4020-9537-455c4e7af78c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: ed75f58f-9135-4d97-b24a-2f8fcdab53ce] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:18.044884 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3c9574f0-719a-4020-9537-455c4e7af78c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:18.045655 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3c9574f0-719a-4020-9537-455c4e7af78c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:18.046277 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3c9574f0-719a-4020-9537-455c4e7af78c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:18.047550 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3c9574f0-719a-4020-9537-455c4e7af78c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:18.053207 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3c9574f0-719a-4020-9537-455c4e7af78c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:18.053207 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3c9574f0-719a-4020-9537-455c4e7af78c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:18.204012 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-3c9574f0-719a-4020-9537-455c4e7af78c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:18.222267 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-3c9574f0-719a-4020-9537-455c4e7af78c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce" status: 200 len: 1522 microversion: 2.1 time: 0.195170 Aug 08 02:27:18.223562 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 119/234] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:18 2022] GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce => generated 1522 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:19.145367 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-b19456a5-c39d-46bd-8d4b-5f5ec2bbee2b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "POST /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.170672 Aug 08 02:27:19.147323 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 116/235] 10.222.0.27 () {60 vars in 1323 bytes} [Mon Aug 8 02:27:17 2022] POST /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e/os-volume_attachments => generated 194 bytes in 1174 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:27:19.257364 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-36f3584a-4d96-437d-8a1c-1c926ff072ef tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:19.262513 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-36f3584a-4d96-437d-8a1c-1c926ff072ef tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: ed75f58f-9135-4d97-b24a-2f8fcdab53ce] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:19.265456 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-36f3584a-4d96-437d-8a1c-1c926ff072ef tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:19.266101 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-36f3584a-4d96-437d-8a1c-1c926ff072ef tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:19.266692 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-36f3584a-4d96-437d-8a1c-1c926ff072ef tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:19.267802 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-36f3584a-4d96-437d-8a1c-1c926ff072ef tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:19.268452 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-36f3584a-4d96-437d-8a1c-1c926ff072ef tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:19.269047 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-36f3584a-4d96-437d-8a1c-1c926ff072ef tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:19.413510 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-36f3584a-4d96-437d-8a1c-1c926ff072ef tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:19.421662 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-36f3584a-4d96-437d-8a1c-1c926ff072ef tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce" status: 200 len: 1502 microversion: 2.1 time: 0.181530 Aug 08 02:27:19.422715 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 120/236] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:19 2022] GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce => generated 1502 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:19.444064 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-b57fdabe-5941-4e7a-aa13-398c2d2f2f20 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "2e8cb0de-e9ac-4d9f-b6d9-a149cfaaaa73"}} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:27:19.446653 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-b57fdabe-5941-4e7a-aa13-398c2d2f2f20 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: ed75f58f-9135-4d97-b24a-2f8fcdab53ce] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:19.453165 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b57fdabe-5941-4e7a-aa13-398c2d2f2f20 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:19.453659 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b57fdabe-5941-4e7a-aa13-398c2d2f2f20 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:19.454098 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b57fdabe-5941-4e7a-aa13-398c2d2f2f20 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:19.454794 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b57fdabe-5941-4e7a-aa13-398c2d2f2f20 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:19.455250 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b57fdabe-5941-4e7a-aa13-398c2d2f2f20 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:19.455677 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b57fdabe-5941-4e7a-aa13-398c2d2f2f20 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:19.478961 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.wsgi [None req-b57fdabe-5941-4e7a-aa13-398c2d2f2f20 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] HTTP exception thrown: Cannot 'attach_volume' instance ed75f58f-9135-4d97-b24a-2f8fcdab53ce while it is in vm_state rescued Aug 08 02:27:19.481167 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-b57fdabe-5941-4e7a-aa13-398c2d2f2f20 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Request method failure captured: Aug 08 02:27:19.481167 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: request: POST /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce/os-volume_attachments HTTP/1.1 Aug 08 02:27:19.481167 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Accept: application/json Aug 08 02:27:19.481167 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Accept-Encoding: identity Aug 08 02:27:19.481167 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Connection: close Aug 08 02:27:19.481167 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Content-Length: 74 Aug 08 02:27:19.481167 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Content-Type: application/json Aug 08 02:27:19.481167 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Host: 10.222.0.27 Aug 08 02:27:19.481167 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Openstack-System-Scope: None Aug 08 02:27:19.481167 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: User-Agent: python-urllib3/1.26.11 Aug 08 02:27:19.481167 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Auth-Token: gAAAAABi8HQXsGK5SLdeRHabROI8DgUVfT8-qt3OmjigKWIwr2r9ylfDom5hOf3PBwXXbVpQKaQQCg2GWwYCANhQNXkemxQtH2Z8vABNoiUbOWpRK-0iPQcLUey3hJs3m48mZ7E_Wt-sxx-vNwT0dxN6KL1UanLlX0GY-h9OUcxTwCxsePW3BgQ Aug 08 02:27:19.481167 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Domain-Id: None Aug 08 02:27:19.481167 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Domain-Name: None Aug 08 02:27:19.481167 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Identity-Status: Confirmed Aug 08 02:27:19.481167 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Is-Admin-Project: True Aug 08 02:27:19.481167 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Domain-Id: default Aug 08 02:27:19.481167 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Domain-Name: Default Aug 08 02:27:19.481167 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Id: 8af7d2bbc6d94273bd39c81423d8cebe Aug 08 02:27:19.481167 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-514896148 Aug 08 02:27:19.481167 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Role: reader,member Aug 08 02:27:19.481167 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Roles: reader,member Aug 08 02:27:19.482681 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/8af7d2bbc6d94273bd39c81423d8cebe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/8af7d2bbc6d94273bd39c81423d8cebe"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_8af7d2bbc6d94273bd39c81423d8cebe", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/8af7d2bbc6d94273bd39c81423d8cebe"}]}] Aug 08 02:27:19.482681 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant: tempest-ServerRescueNegativeTestJSON-514896148 Aug 08 02:27:19.482681 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant-Id: 8af7d2bbc6d94273bd39c81423d8cebe Aug 08 02:27:19.482681 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-514896148 Aug 08 02:27:19.482681 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User: tempest-ServerRescueNegativeTestJSON-514896148-project Aug 08 02:27:19.482681 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Domain-Id: default Aug 08 02:27:19.482681 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Domain-Name: Default Aug 08 02:27:19.482681 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Id: 5c8ae69712c14878bd2aab6d33512b7a Aug 08 02:27:19.482681 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Name: tempest-ServerRescueNegativeTestJSON-514896148-project Aug 08 02:27:19.482681 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Aug 08 02:27:19.482681 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: {"volumeAttachment": {"volumeId": "2e8cb0de-e9ac-4d9f-b6d9-a149cfaaaa73"}} Aug 08 02:27:19.482681 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: method: > Aug 08 02:27:19.485527 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: exception: Cannot 'attach_volume' instance ed75f58f-9135-4d97-b24a-2f8fcdab53ce while it is in vm_state rescued Aug 08 02:27:19.485527 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: action_args: {'server_id': 'ed75f58f-9135-4d97-b24a-2f8fcdab53ce', 'body': {'volumeAttachment': {'volumeId': '2e8cb0de-e9ac-4d9f-b6d9-a149cfaaaa73'}}} Aug 08 02:27:19.485527 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:27:19.485527 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:27:19.485527 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Aug 08 02:27:19.485527 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Aug 08 02:27:19.485527 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 08 02:27:19.485527 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 08 02:27:19.485527 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner Aug 08 02:27:19.485527 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Aug 08 02:27:19.485527 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance ed75f58f-9135-4d97-b24a-2f8fcdab53ce in vm_state rescued. Cannot attach_volume while the instance is in this state. Aug 08 02:27:19.485527 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:27:19.485527 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:27:19.485527 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:27:19.485527 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:27:19.485527 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:27:19.485527 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:27:19.485527 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:27:19.485527 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:27:19.486896 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:27:19.486896 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:27:19.486896 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 08 02:27:19.486896 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 08 02:27:19.486896 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 08 02:27:19.486896 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 08 02:27:19.486896 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 08 02:27:19.486896 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 08 02:27:19.486896 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 398, in create Aug 08 02:27:19.486896 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Aug 08 02:27:19.486896 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Aug 08 02:27:19.486896 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Aug 08 02:27:19.486896 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'attach_volume' instance ed75f58f-9135-4d97-b24a-2f8fcdab53ce while it is in vm_state rescued Aug 08 02:27:19.486896 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:27:19.486896 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:27:19.486896 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:27:19.486896 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:27:19.486896 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:27:19.486896 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:27:19.488568 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:27:19.488568 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:27:19.488568 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'attach_volume' instance ed75f58f-9135-4d97-b24a-2f8fcdab53ce while it is in vm_state rescued Aug 08 02:27:19.488568 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:27:19.488568 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-b57fdabe-5941-4e7a-aa13-398c2d2f2f20 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Returning 409 to user: Cannot 'attach_volume' instance ed75f58f-9135-4d97-b24a-2f8fcdab53ce while it is in vm_state rescued {{(pid=118351) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:27:19.488568 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-b57fdabe-5941-4e7a-aa13-398c2d2f2f20 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "POST /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce/os-volume_attachments" status: 409 len: 152 microversion: 2.1 time: 0.043237 Aug 08 02:27:19.488568 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 117/237] 10.222.0.27 () {60 vars in 1323 bytes} [Mon Aug 8 02:27:19 2022] POST /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce/os-volume_attachments => generated 152 bytes in 45 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Aug 08 02:27:19.492695 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-28f3cbf1-7fba-4b8d-8215-048fb792ce24 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:27:19.493220 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-28f3cbf1-7fba-4b8d-8215-048fb792ce24 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: ed75f58f-9135-4d97-b24a-2f8fcdab53ce] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:19.500429 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-28f3cbf1-7fba-4b8d-8215-048fb792ce24 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:19.500990 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-28f3cbf1-7fba-4b8d-8215-048fb792ce24 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:19.501429 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-28f3cbf1-7fba-4b8d-8215-048fb792ce24 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:19.502116 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-28f3cbf1-7fba-4b8d-8215-048fb792ce24 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:19.502650 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-28f3cbf1-7fba-4b8d-8215-048fb792ce24 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:19.503261 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-28f3cbf1-7fba-4b8d-8215-048fb792ce24 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:19.798466 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-28f3cbf1-7fba-4b8d-8215-048fb792ce24 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "POST /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce/action" status: 202 len: 0 microversion: 2.1 time: 0.309393 Aug 08 02:27:19.802488 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 121/238] 10.222.0.27 () {60 vars in 1278 bytes} [Mon Aug 8 02:27:19 2022] POST /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce/action => generated 0 bytes in 311 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 08 02:27:19.807542 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-4fe1cec9-c68f-4a3c-88cd-7605f5c031f5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:19.808157 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-4fe1cec9-c68f-4a3c-88cd-7605f5c031f5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: ed75f58f-9135-4d97-b24a-2f8fcdab53ce] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:19.815671 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4fe1cec9-c68f-4a3c-88cd-7605f5c031f5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:19.816159 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4fe1cec9-c68f-4a3c-88cd-7605f5c031f5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:19.816596 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4fe1cec9-c68f-4a3c-88cd-7605f5c031f5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:19.817330 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4fe1cec9-c68f-4a3c-88cd-7605f5c031f5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:19.817783 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4fe1cec9-c68f-4a3c-88cd-7605f5c031f5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:19.818217 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4fe1cec9-c68f-4a3c-88cd-7605f5c031f5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:19.960595 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-4fe1cec9-c68f-4a3c-88cd-7605f5c031f5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:19.970304 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-4fe1cec9-c68f-4a3c-88cd-7605f5c031f5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce" status: 200 len: 1510 microversion: 2.1 time: 0.166474 Aug 08 02:27:19.972848 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 118/239] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:19 2022] GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce => generated 1510 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:20.994345 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-dfecba1b-7e68-4d0e-ad62-84324cd57334 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:20.996368 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-dfecba1b-7e68-4d0e-ad62-84324cd57334 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: ed75f58f-9135-4d97-b24a-2f8fcdab53ce] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:21.007253 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dfecba1b-7e68-4d0e-ad62-84324cd57334 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:21.007253 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dfecba1b-7e68-4d0e-ad62-84324cd57334 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:21.007253 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dfecba1b-7e68-4d0e-ad62-84324cd57334 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:21.007253 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dfecba1b-7e68-4d0e-ad62-84324cd57334 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:21.007616 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dfecba1b-7e68-4d0e-ad62-84324cd57334 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:21.010601 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dfecba1b-7e68-4d0e-ad62-84324cd57334 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:21.031730 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:21.151309 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-dfecba1b-7e68-4d0e-ad62-84324cd57334 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:21.160006 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-dfecba1b-7e68-4d0e-ad62-84324cd57334 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce" status: 200 len: 1510 microversion: 2.1 time: 0.172182 Aug 08 02:27:21.160945 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 122/240] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:20 2022] GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce => generated 1510 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:22.182408 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-d792b06f-4eee-470b-b779-5efeaabdc1de tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:22.183074 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-d792b06f-4eee-470b-b779-5efeaabdc1de tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: ed75f58f-9135-4d97-b24a-2f8fcdab53ce] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:22.191296 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d792b06f-4eee-470b-b779-5efeaabdc1de tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:22.191856 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d792b06f-4eee-470b-b779-5efeaabdc1de tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:22.192449 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d792b06f-4eee-470b-b779-5efeaabdc1de tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:22.193349 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d792b06f-4eee-470b-b779-5efeaabdc1de tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:22.193934 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d792b06f-4eee-470b-b779-5efeaabdc1de tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:22.194367 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d792b06f-4eee-470b-b779-5efeaabdc1de tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:22.373090 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-d792b06f-4eee-470b-b779-5efeaabdc1de tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:22.381328 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-d792b06f-4eee-470b-b779-5efeaabdc1de tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce" status: 200 len: 1510 microversion: 2.1 time: 0.202930 Aug 08 02:27:22.381906 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 119/241] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:22 2022] GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce => generated 1510 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:23.406136 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-473b0c4e-3c0e-4fba-90be-09527a8531bb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:23.406995 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-473b0c4e-3c0e-4fba-90be-09527a8531bb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: ed75f58f-9135-4d97-b24a-2f8fcdab53ce] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:23.414688 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-473b0c4e-3c0e-4fba-90be-09527a8531bb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:23.417444 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-473b0c4e-3c0e-4fba-90be-09527a8531bb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:23.417444 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-473b0c4e-3c0e-4fba-90be-09527a8531bb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:23.417444 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-473b0c4e-3c0e-4fba-90be-09527a8531bb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:23.417444 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-473b0c4e-3c0e-4fba-90be-09527a8531bb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:23.417845 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-473b0c4e-3c0e-4fba-90be-09527a8531bb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:23.540155 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-25fb5489-5df0-487f-bb4b-3f45a1c197d9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:23.541005 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-25fb5489-5df0-487f-bb4b-3f45a1c197d9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:23.552177 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-25fb5489-5df0-487f-bb4b-3f45a1c197d9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:23.552906 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-25fb5489-5df0-487f-bb4b-3f45a1c197d9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:23.554766 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-25fb5489-5df0-487f-bb4b-3f45a1c197d9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:23.566333 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-25fb5489-5df0-487f-bb4b-3f45a1c197d9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:23.566333 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-25fb5489-5df0-487f-bb4b-3f45a1c197d9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:23.566333 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-25fb5489-5df0-487f-bb4b-3f45a1c197d9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:23.606578 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-25fb5489-5df0-487f-bb4b-3f45a1c197d9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091/ips" status: 200 len: 152 microversion: 2.20 time: 0.073101 Aug 08 02:27:23.607424 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 120/242] 10.222.0.27 () {60 vars in 1289 bytes} [Mon Aug 8 02:27:23 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091/ips => generated 152 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 08 02:27:23.739907 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-473b0c4e-3c0e-4fba-90be-09527a8531bb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:23.747836 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-473b0c4e-3c0e-4fba-90be-09527a8531bb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce" status: 200 len: 1516 microversion: 2.1 time: 0.346337 Aug 08 02:27:23.749726 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 123/243] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:23 2022] GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce => generated 1516 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:26.191973 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-eaf78cb7-1b55-4090-8528-5acdda5ebdec tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Action: 'create', calling method: .version_select at 0x7efc74367790>, body: {"keypair": {"name": "tempest-keypair-2060462698", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBL/dnMHI6yf1BFDLOrOk/9i6zrA/ASXramzae94V2d/7tCxGaN5pcxVl+f0STdcVaZ0almQEY2uNksrJ7p6J0bpJTWHNc8UZ2WLScGjQWfuON37nv8a/LzzkD+Qjdml7iw=="}} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:27:26.213639 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.quota [None req-eaf78cb7-1b55-4090-8528-5acdda5ebdec tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Getting quotas for project 8af7d2bbc6d94273bd39c81423d8cebe. Resources: {'key_pairs'} {{(pid=118351) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 08 02:27:26.219529 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.quota [None req-eaf78cb7-1b55-4090-8528-5acdda5ebdec tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Getting quotas for user 5c8ae69712c14878bd2aab6d33512b7a and project 8af7d2bbc6d94273bd39c81423d8cebe. Resources: {'key_pairs'} {{(pid=118351) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 08 02:27:26.324040 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-eaf78cb7-1b55-4090-8528-5acdda5ebdec tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.137076 Aug 08 02:27:26.324660 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 121/244] 10.222.0.27 () {60 vars in 1159 bytes} [Mon Aug 8 02:27:26 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:27:26.325583 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-a9dcc7d7-bd09-4e28-beee-12abb88b1c87 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:27:26.337453 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a9dcc7d7-bd09-4e28-beee-12abb88b1c87 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:26.338684 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a9dcc7d7-bd09-4e28-beee-12abb88b1c87 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:26.339671 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a9dcc7d7-bd09-4e28-beee-12abb88b1c87 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:26.408139 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-a9dcc7d7-bd09-4e28-beee-12abb88b1c87 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Going to try to stop instance {{(pid=118352) force_stop /opt/stack/nova/nova/compute/api.py:2709}} Aug 08 02:27:26.610154 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-5e29b022-6d1f-4c6c-9eab-7fc97769eda9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:27:26.610987 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-5e29b022-6d1f-4c6c-9eab-7fc97769eda9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:26.619852 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5e29b022-6d1f-4c6c-9eab-7fc97769eda9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:26.620449 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5e29b022-6d1f-4c6c-9eab-7fc97769eda9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:26.621006 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5e29b022-6d1f-4c6c-9eab-7fc97769eda9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:26.621538 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5e29b022-6d1f-4c6c-9eab-7fc97769eda9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:26.622207 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5e29b022-6d1f-4c6c-9eab-7fc97769eda9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:26.622985 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5e29b022-6d1f-4c6c-9eab-7fc97769eda9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:26.648871 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.objects.instance [None req-5e29b022-6d1f-4c6c-9eab-7fc97769eda9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lazy-loading 'flavor' on Instance uuid f51a5900-5abc-4188-bbc4-33b0f0f7d091 {{(pid=118351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 08 02:27:26.670435 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-a9dcc7d7-bd09-4e28-beee-12abb88b1c87 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "POST /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a/action" status: 202 len: 0 microversion: 2.1 time: 0.348696 Aug 08 02:27:26.671191 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 124/245] 10.222.0.27 () {60 vars in 1278 bytes} [Mon Aug 8 02:27:26 2022] POST /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a/action => generated 0 bytes in 351 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 08 02:27:26.680453 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-37ebdbb5-9259-4bcf-a255-9ed2c0ed8a34 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:26.681173 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-37ebdbb5-9259-4bcf-a255-9ed2c0ed8a34 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:26.688109 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-37ebdbb5-9259-4bcf-a255-9ed2c0ed8a34 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:26.688883 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-37ebdbb5-9259-4bcf-a255-9ed2c0ed8a34 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:26.689322 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-37ebdbb5-9259-4bcf-a255-9ed2c0ed8a34 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:26.690006 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-37ebdbb5-9259-4bcf-a255-9ed2c0ed8a34 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:26.690450 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-37ebdbb5-9259-4bcf-a255-9ed2c0ed8a34 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:26.690870 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-37ebdbb5-9259-4bcf-a255-9ed2c0ed8a34 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:26.876119 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-37ebdbb5-9259-4bcf-a255-9ed2c0ed8a34 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:26.884730 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-37ebdbb5-9259-4bcf-a255-9ed2c0ed8a34 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1726 microversion: 2.1 time: 0.208302 Aug 08 02:27:26.885511 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 125/246] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:26 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1726 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:27:27.260420 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-5e29b022-6d1f-4c6c-9eab-7fc97769eda9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "POST /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091/action" status: 202 len: 0 microversion: 2.20 time: 0.656631 Aug 08 02:27:27.261373 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 122/247] 10.222.0.27 () {62 vars in 1319 bytes} [Mon Aug 8 02:27:26 2022] POST /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091/action => generated 0 bytes in 658 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Aug 08 02:27:27.269902 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-3950cc3b-490f-4a45-b91a-8d4fa1faaee5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:27.270508 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-3950cc3b-490f-4a45-b91a-8d4fa1faaee5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:27.285300 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3950cc3b-490f-4a45-b91a-8d4fa1faaee5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:27.285992 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3950cc3b-490f-4a45-b91a-8d4fa1faaee5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:27.286442 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3950cc3b-490f-4a45-b91a-8d4fa1faaee5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:27.287246 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3950cc3b-490f-4a45-b91a-8d4fa1faaee5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:27.287808 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3950cc3b-490f-4a45-b91a-8d4fa1faaee5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:27.293932 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3950cc3b-490f-4a45-b91a-8d4fa1faaee5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:27.654642 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-3950cc3b-490f-4a45-b91a-8d4fa1faaee5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:27.664322 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-3950cc3b-490f-4a45-b91a-8d4fa1faaee5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:27.666349 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-3950cc3b-490f-4a45-b91a-8d4fa1faaee5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:27.667899 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-3950cc3b-490f-4a45-b91a-8d4fa1faaee5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1725 microversion: 2.20 time: 0.401811 Aug 08 02:27:27.668486 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 126/248] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:27:27 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1725 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:27:27.908230 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-b639513e-41ab-4be5-ab8d-5935d8c6bd63 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:27.908973 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-b639513e-41ab-4be5-ab8d-5935d8c6bd63 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:27.916323 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b639513e-41ab-4be5-ab8d-5935d8c6bd63 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:27.916873 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b639513e-41ab-4be5-ab8d-5935d8c6bd63 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:27.917315 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b639513e-41ab-4be5-ab8d-5935d8c6bd63 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:27.918020 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b639513e-41ab-4be5-ab8d-5935d8c6bd63 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:27.918475 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b639513e-41ab-4be5-ab8d-5935d8c6bd63 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:27.918901 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b639513e-41ab-4be5-ab8d-5935d8c6bd63 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:28.132173 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-b639513e-41ab-4be5-ab8d-5935d8c6bd63 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:28.141145 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-b639513e-41ab-4be5-ab8d-5935d8c6bd63 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1726 microversion: 2.1 time: 0.237684 Aug 08 02:27:28.141763 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 123/249] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:27 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1726 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:28.691796 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-979efa5e-2ec1-45f2-ba13-34e64eceeab9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:28.692436 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-979efa5e-2ec1-45f2-ba13-34e64eceeab9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:28.699041 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-979efa5e-2ec1-45f2-ba13-34e64eceeab9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:28.699755 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-979efa5e-2ec1-45f2-ba13-34e64eceeab9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:28.700195 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-979efa5e-2ec1-45f2-ba13-34e64eceeab9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:28.700941 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-979efa5e-2ec1-45f2-ba13-34e64eceeab9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:28.704276 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-979efa5e-2ec1-45f2-ba13-34e64eceeab9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:28.704276 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-979efa5e-2ec1-45f2-ba13-34e64eceeab9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:28.841320 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-979efa5e-2ec1-45f2-ba13-34e64eceeab9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:28.851229 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-979efa5e-2ec1-45f2-ba13-34e64eceeab9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:28.856055 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-979efa5e-2ec1-45f2-ba13-34e64eceeab9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:28.857484 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-979efa5e-2ec1-45f2-ba13-34e64eceeab9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1725 microversion: 2.20 time: 0.169948 Aug 08 02:27:28.873011 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 127/250] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:27:28 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1725 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:27:29.163427 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-bfffa994-a1fe-40d6-8c6d-8acf36b50488 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:29.164061 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-bfffa994-a1fe-40d6-8c6d-8acf36b50488 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:29.170962 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-bfffa994-a1fe-40d6-8c6d-8acf36b50488 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:29.172029 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-bfffa994-a1fe-40d6-8c6d-8acf36b50488 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:29.172537 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-bfffa994-a1fe-40d6-8c6d-8acf36b50488 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:29.173263 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-bfffa994-a1fe-40d6-8c6d-8acf36b50488 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:29.173711 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-bfffa994-a1fe-40d6-8c6d-8acf36b50488 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:29.174752 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-bfffa994-a1fe-40d6-8c6d-8acf36b50488 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:29.329650 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-bfffa994-a1fe-40d6-8c6d-8acf36b50488 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:29.338196 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-bfffa994-a1fe-40d6-8c6d-8acf36b50488 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1726 microversion: 2.1 time: 0.178911 Aug 08 02:27:29.338760 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 124/251] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:29 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1726 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:29.695880 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-8942e84c-cc9e-4593-b996-40b919fa5b2a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-2027421855", "imageRef": "30833f47-5987-41ca-b5ee-ad5ecafbb265", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "847f3ad1-eef7-41cf-8570-7f9863393002"}], "security_groups": [{"name": "tempest-securitygroup--1934562080"}], "key_name": "tempest-keypair-2060462698", "user_data": "}} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:27:29.820105 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.network.neutron [None req-8942e84c-cc9e-4593-b996-40b919fa5b2a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] validate_networks() for [('847f3ad1-eef7-41cf-8570-7f9863393002', None, None, None, None, None)] {{(pid=118352) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 08 02:27:29.884411 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-3e4f3e12-b623-443d-8b1f-f98b74b46d45 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:29.885089 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-3e4f3e12-b623-443d-8b1f-f98b74b46d45 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:29.891988 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3e4f3e12-b623-443d-8b1f-f98b74b46d45 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:29.892475 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3e4f3e12-b623-443d-8b1f-f98b74b46d45 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:29.892948 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3e4f3e12-b623-443d-8b1f-f98b74b46d45 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:29.893641 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3e4f3e12-b623-443d-8b1f-f98b74b46d45 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:29.894083 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3e4f3e12-b623-443d-8b1f-f98b74b46d45 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:29.894527 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3e4f3e12-b623-443d-8b1f-f98b74b46d45 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:30.173053 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-3e4f3e12-b623-443d-8b1f-f98b74b46d45 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:30.182592 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-3e4f3e12-b623-443d-8b1f-f98b74b46d45 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:30.190052 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-3e4f3e12-b623-443d-8b1f-f98b74b46d45 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:30.191255 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-3e4f3e12-b623-443d-8b1f-f98b74b46d45 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1725 microversion: 2.20 time: 0.312443 Aug 08 02:27:30.192120 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 125/252] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:27:29 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1725 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:27:30.328946 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-8942e84c-cc9e-4593-b996-40b919fa5b2a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Flavor limits 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 08 02:27:30.328946 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-8942e84c-cc9e-4593-b996-40b919fa5b2a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Image limits 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 08 02:27:30.328946 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-8942e84c-cc9e-4593-b996-40b919fa5b2a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Flavor pref 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 08 02:27:30.328946 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-8942e84c-cc9e-4593-b996-40b919fa5b2a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Image pref 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 08 02:27:30.328946 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-8942e84c-cc9e-4593-b996-40b919fa5b2a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 08 02:27:30.335094 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8942e84c-cc9e-4593-b996-40b919fa5b2a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:30.335581 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8942e84c-cc9e-4593-b996-40b919fa5b2a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:30.336007 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8942e84c-cc9e-4593-b996-40b919fa5b2a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:30.336823 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8942e84c-cc9e-4593-b996-40b919fa5b2a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:30.337282 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8942e84c-cc9e-4593-b996-40b919fa5b2a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:30.337708 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8942e84c-cc9e-4593-b996-40b919fa5b2a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:30.356775 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.quota [None req-8942e84c-cc9e-4593-b996-40b919fa5b2a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Getting quotas for project 8af7d2bbc6d94273bd39c81423d8cebe. Resources: {'ram', 'cores', 'instances'} {{(pid=118352) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 08 02:27:30.360589 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-2ee4af8d-91b0-4e95-91d9-334a31ddbac4 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:30.361488 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-2ee4af8d-91b0-4e95-91d9-334a31ddbac4 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:30.361810 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.quota [None req-8942e84c-cc9e-4593-b996-40b919fa5b2a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Getting quotas for user 5c8ae69712c14878bd2aab6d33512b7a and project 8af7d2bbc6d94273bd39c81423d8cebe. Resources: {'ram', 'cores', 'instances'} {{(pid=118352) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 08 02:27:30.367565 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2ee4af8d-91b0-4e95-91d9-334a31ddbac4 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:30.368137 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2ee4af8d-91b0-4e95-91d9-334a31ddbac4 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:30.368940 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2ee4af8d-91b0-4e95-91d9-334a31ddbac4 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:30.369641 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2ee4af8d-91b0-4e95-91d9-334a31ddbac4 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:30.370100 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2ee4af8d-91b0-4e95-91d9-334a31ddbac4 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:30.370531 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2ee4af8d-91b0-4e95-91d9-334a31ddbac4 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:30.373949 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-8942e84c-cc9e-4593-b996-40b919fa5b2a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Going to run 1 instances... {{(pid=118352) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Aug 08 02:27:30.376498 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-8942e84c-cc9e-4593-b996-40b919fa5b2a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='30833f47-5987-41ca-b5ee-ad5ecafbb265',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118352) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Aug 08 02:27:30.524874 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-2ee4af8d-91b0-4e95-91d9-334a31ddbac4 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:30.533419 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-2ee4af8d-91b0-4e95-91d9-334a31ddbac4 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1726 microversion: 2.1 time: 0.177890 Aug 08 02:27:30.533992 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 126/253] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:30 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1726 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:27:30.554562 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-8942e84c-cc9e-4593-b996-40b919fa5b2a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 409 microversion: 2.1 time: 0.863666 Aug 08 02:27:30.556830 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 128/254] 10.222.0.27 () {60 vars in 1147 bytes} [Mon Aug 8 02:27:29 2022] POST /compute/v2.1/servers => generated 409 bytes in 865 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Aug 08 02:27:30.566985 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-7ca07da5-0740-41d4-aa77-54b3f89f9a59 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:30.567676 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-7ca07da5-0740-41d4-aa77-54b3f89f9a59 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:30.632223 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-7ca07da5-0740-41d4-aa77-54b3f89f9a59 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:30.641096 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-7ca07da5-0740-41d4-aa77-54b3f89f9a59 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1246 microversion: 2.1 time: 0.076510 Aug 08 02:27:30.641652 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 127/255] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:30 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1246 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:31.220197 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-70b6e4ac-c18f-40a9-a63a-8386275895d5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:31.220840 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-70b6e4ac-c18f-40a9-a63a-8386275895d5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:31.228209 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-70b6e4ac-c18f-40a9-a63a-8386275895d5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:31.228881 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-70b6e4ac-c18f-40a9-a63a-8386275895d5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:31.229420 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-70b6e4ac-c18f-40a9-a63a-8386275895d5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:31.230277 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-70b6e4ac-c18f-40a9-a63a-8386275895d5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:31.230665 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-70b6e4ac-c18f-40a9-a63a-8386275895d5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:31.231098 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-70b6e4ac-c18f-40a9-a63a-8386275895d5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:31.379597 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-70b6e4ac-c18f-40a9-a63a-8386275895d5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:31.388895 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-70b6e4ac-c18f-40a9-a63a-8386275895d5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:31.390957 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-70b6e4ac-c18f-40a9-a63a-8386275895d5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:31.392268 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-70b6e4ac-c18f-40a9-a63a-8386275895d5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1725 microversion: 2.20 time: 0.176542 Aug 08 02:27:31.392792 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 129/256] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:27:31 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1725 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:27:31.556567 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-a960efdf-08d0-4038-95eb-154462618898 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:31.557210 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-a960efdf-08d0-4038-95eb-154462618898 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:31.565968 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a960efdf-08d0-4038-95eb-154462618898 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:31.566463 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a960efdf-08d0-4038-95eb-154462618898 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:31.566890 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a960efdf-08d0-4038-95eb-154462618898 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:31.567593 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a960efdf-08d0-4038-95eb-154462618898 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:31.568040 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a960efdf-08d0-4038-95eb-154462618898 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:31.572254 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a960efdf-08d0-4038-95eb-154462618898 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:31.664599 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-3e11b1e2-3bed-4ddd-bcb5-b6ed2fae9752 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:31.664599 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-3e11b1e2-3bed-4ddd-bcb5-b6ed2fae9752 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:31.671079 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3e11b1e2-3bed-4ddd-bcb5-b6ed2fae9752 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:31.672141 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3e11b1e2-3bed-4ddd-bcb5-b6ed2fae9752 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:31.672812 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3e11b1e2-3bed-4ddd-bcb5-b6ed2fae9752 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:31.673536 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3e11b1e2-3bed-4ddd-bcb5-b6ed2fae9752 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:31.674225 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3e11b1e2-3bed-4ddd-bcb5-b6ed2fae9752 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:31.674920 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3e11b1e2-3bed-4ddd-bcb5-b6ed2fae9752 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:31.730099 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-a960efdf-08d0-4038-95eb-154462618898 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:31.738889 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-a960efdf-08d0-4038-95eb-154462618898 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1726 microversion: 2.1 time: 0.186100 Aug 08 02:27:31.739468 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 128/257] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:31 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1726 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:31.755610 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-3e11b1e2-3bed-4ddd-bcb5-b6ed2fae9752 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:31.763354 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-3e11b1e2-3bed-4ddd-bcb5-b6ed2fae9752 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1250 microversion: 2.1 time: 0.104909 Aug 08 02:27:31.765967 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 130/258] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:31 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1250 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:32.418581 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-c0a1ff76-4de8-463f-bb59-fd27f7dbdbcc tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:32.419209 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-c0a1ff76-4de8-463f-bb59-fd27f7dbdbcc tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:32.427280 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c0a1ff76-4de8-463f-bb59-fd27f7dbdbcc tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:32.427779 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c0a1ff76-4de8-463f-bb59-fd27f7dbdbcc tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:32.428235 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c0a1ff76-4de8-463f-bb59-fd27f7dbdbcc tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:32.428959 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c0a1ff76-4de8-463f-bb59-fd27f7dbdbcc tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:32.429418 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c0a1ff76-4de8-463f-bb59-fd27f7dbdbcc tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:32.429862 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c0a1ff76-4de8-463f-bb59-fd27f7dbdbcc tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:32.574327 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-c0a1ff76-4de8-463f-bb59-fd27f7dbdbcc tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:32.578660 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=4,nova_cell1:INSERT=2 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:32.585317 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-c0a1ff76-4de8-463f-bb59-fd27f7dbdbcc tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:32.587015 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-c0a1ff76-4de8-463f-bb59-fd27f7dbdbcc tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:32.588598 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-c0a1ff76-4de8-463f-bb59-fd27f7dbdbcc tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1725 microversion: 2.20 time: 0.173678 Aug 08 02:27:32.588856 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 129/259] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:27:32 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1725 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 08 02:27:32.761006 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-568f5984-3ae5-4d7d-b5b7-b23fed03020d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:32.762207 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-568f5984-3ae5-4d7d-b5b7-b23fed03020d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:32.769037 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-568f5984-3ae5-4d7d-b5b7-b23fed03020d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:32.769443 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-568f5984-3ae5-4d7d-b5b7-b23fed03020d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:32.774735 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-568f5984-3ae5-4d7d-b5b7-b23fed03020d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:32.774735 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-568f5984-3ae5-4d7d-b5b7-b23fed03020d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:32.774735 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-568f5984-3ae5-4d7d-b5b7-b23fed03020d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:32.774735 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-568f5984-3ae5-4d7d-b5b7-b23fed03020d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:32.783915 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-7776c96f-af8a-4572-aede-ef74972cd8ef tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:32.784528 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-7776c96f-af8a-4572-aede-ef74972cd8ef tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:32.792529 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7776c96f-af8a-4572-aede-ef74972cd8ef tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:32.793056 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7776c96f-af8a-4572-aede-ef74972cd8ef tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:32.793794 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7776c96f-af8a-4572-aede-ef74972cd8ef tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:32.794473 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7776c96f-af8a-4572-aede-ef74972cd8ef tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:32.794929 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7776c96f-af8a-4572-aede-ef74972cd8ef tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:32.795353 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7776c96f-af8a-4572-aede-ef74972cd8ef tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:32.871728 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-7776c96f-af8a-4572-aede-ef74972cd8ef tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:32.881403 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-7776c96f-af8a-4572-aede-ef74972cd8ef tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1242 microversion: 2.1 time: 0.101321 Aug 08 02:27:32.893211 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 130/260] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:32 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1242 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:27:32.950761 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-568f5984-3ae5-4d7d-b5b7-b23fed03020d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:32.959524 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-568f5984-3ae5-4d7d-b5b7-b23fed03020d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1726 microversion: 2.1 time: 0.202297 Aug 08 02:27:32.960128 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 131/261] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:32 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1726 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:33.612141 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-354e591e-4298-426e-8d13-837409c8f4e8 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:33.613471 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-354e591e-4298-426e-8d13-837409c8f4e8 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:33.621618 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-354e591e-4298-426e-8d13-837409c8f4e8 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:33.622099 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-354e591e-4298-426e-8d13-837409c8f4e8 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:33.622836 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-354e591e-4298-426e-8d13-837409c8f4e8 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:33.623697 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-354e591e-4298-426e-8d13-837409c8f4e8 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:33.624815 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-354e591e-4298-426e-8d13-837409c8f4e8 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:33.624815 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-354e591e-4298-426e-8d13-837409c8f4e8 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:33.795427 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-354e591e-4298-426e-8d13-837409c8f4e8 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:33.804716 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-354e591e-4298-426e-8d13-837409c8f4e8 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:33.806286 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-354e591e-4298-426e-8d13-837409c8f4e8 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:33.807477 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-354e591e-4298-426e-8d13-837409c8f4e8 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1725 microversion: 2.20 time: 0.199745 Aug 08 02:27:33.808256 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 131/262] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:27:33 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1725 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:27:33.824852 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-ea3f85e7-fd17-4c69-afb7-f0d3d8735453 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Action: 'create', calling method: .version_select at 0x7efc743ef160>, body: {"keypair": {"name": "tempest-keypair-458871973", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBIShmVAn6Z8A4Ty1Qty55soSQhEHl1LFHmL/jW0/mahT+JKnLb3rlerrZ7DJbpLH4RM9h1eMnixIgVBjPHOuFzVb9Wh2mF1PmvZ2QYBcrsANh/kuSXNY0ulirShwVhDYBg=="}} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:27:33.845541 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.quota [None req-ea3f85e7-fd17-4c69-afb7-f0d3d8735453 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Getting quotas for project ca29b2b9a59d4826bfd25f4dcfa9cffe. Resources: {'key_pairs'} {{(pid=118352) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 08 02:27:33.850427 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.quota [None req-ea3f85e7-fd17-4c69-afb7-f0d3d8735453 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Getting quotas for user 578cd837d9fe442587f8c6c8528334c0 and project ca29b2b9a59d4826bfd25f4dcfa9cffe. Resources: {'key_pairs'} {{(pid=118352) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 08 02:27:33.903608 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-09f56c7c-6e51-4a6d-84ca-74417edf0a96 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:33.904299 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-09f56c7c-6e51-4a6d-84ca-74417edf0a96 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:33.910696 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-09f56c7c-6e51-4a6d-84ca-74417edf0a96 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:33.911209 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-09f56c7c-6e51-4a6d-84ca-74417edf0a96 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:33.911643 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-09f56c7c-6e51-4a6d-84ca-74417edf0a96 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:33.912382 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-09f56c7c-6e51-4a6d-84ca-74417edf0a96 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:33.912914 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-09f56c7c-6e51-4a6d-84ca-74417edf0a96 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:33.913381 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-09f56c7c-6e51-4a6d-84ca-74417edf0a96 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:33.935936 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-ea3f85e7-fd17-4c69-afb7-f0d3d8735453 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.115778 Aug 08 02:27:33.937738 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 132/263] 10.222.0.27 () {60 vars in 1159 bytes} [Mon Aug 8 02:27:33 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:27:33.982264 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-c96d3c8f-aae9-4ed4-bf45-c4bbc07c2b6e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:33.982990 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-c96d3c8f-aae9-4ed4-bf45-c4bbc07c2b6e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:33.991406 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c96d3c8f-aae9-4ed4-bf45-c4bbc07c2b6e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:33.992024 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c96d3c8f-aae9-4ed4-bf45-c4bbc07c2b6e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:33.992724 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c96d3c8f-aae9-4ed4-bf45-c4bbc07c2b6e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:33.993376 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c96d3c8f-aae9-4ed4-bf45-c4bbc07c2b6e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:33.993842 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c96d3c8f-aae9-4ed4-bf45-c4bbc07c2b6e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:33.994303 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c96d3c8f-aae9-4ed4-bf45-c4bbc07c2b6e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:33.994835 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-09f56c7c-6e51-4a6d-84ca-74417edf0a96 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:34.004291 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-09f56c7c-6e51-4a6d-84ca-74417edf0a96 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1316 microversion: 2.1 time: 0.104818 Aug 08 02:27:34.004902 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 132/264] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:33 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1316 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:27:34.169666 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-c96d3c8f-aae9-4ed4-bf45-c4bbc07c2b6e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:34.179569 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-c96d3c8f-aae9-4ed4-bf45-c4bbc07c2b6e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1726 microversion: 2.1 time: 0.201605 Aug 08 02:27:34.180187 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 133/265] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:33 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1726 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:27:34.835651 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-84484ffb-9bbc-439e-87d7-4a29be33020a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:34.836249 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-84484ffb-9bbc-439e-87d7-4a29be33020a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:34.843367 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-84484ffb-9bbc-439e-87d7-4a29be33020a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:34.843938 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-84484ffb-9bbc-439e-87d7-4a29be33020a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:34.844533 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-84484ffb-9bbc-439e-87d7-4a29be33020a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:34.846285 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-84484ffb-9bbc-439e-87d7-4a29be33020a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:34.846285 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-84484ffb-9bbc-439e-87d7-4a29be33020a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:34.846285 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-84484ffb-9bbc-439e-87d7-4a29be33020a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:34.990619 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-84484ffb-9bbc-439e-87d7-4a29be33020a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:35.001058 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-84484ffb-9bbc-439e-87d7-4a29be33020a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:35.002946 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-84484ffb-9bbc-439e-87d7-4a29be33020a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:35.004525 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-84484ffb-9bbc-439e-87d7-4a29be33020a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1725 microversion: 2.20 time: 0.172757 Aug 08 02:27:35.005612 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 133/266] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:27:34 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1725 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:27:35.031535 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-33c1dbde-53a9-4ffb-a35a-c74ad0ec5ee5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:35.032860 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-33c1dbde-53a9-4ffb-a35a-c74ad0ec5ee5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:35.040165 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-33c1dbde-53a9-4ffb-a35a-c74ad0ec5ee5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:35.040761 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-33c1dbde-53a9-4ffb-a35a-c74ad0ec5ee5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:35.041229 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-33c1dbde-53a9-4ffb-a35a-c74ad0ec5ee5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:35.042149 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-33c1dbde-53a9-4ffb-a35a-c74ad0ec5ee5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:35.042919 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-33c1dbde-53a9-4ffb-a35a-c74ad0ec5ee5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:35.043500 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-33c1dbde-53a9-4ffb-a35a-c74ad0ec5ee5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:35.124971 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-33c1dbde-53a9-4ffb-a35a-c74ad0ec5ee5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:35.133144 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-33c1dbde-53a9-4ffb-a35a-c74ad0ec5ee5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1304 microversion: 2.1 time: 0.106900 Aug 08 02:27:35.144992 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 134/267] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:35 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1304 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:35.202827 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-d99b0b4c-4c0b-4035-95c2-4bfa02a97d47 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:35.203456 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-d99b0b4c-4c0b-4035-95c2-4bfa02a97d47 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:35.209469 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d99b0b4c-4c0b-4035-95c2-4bfa02a97d47 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:35.209951 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d99b0b4c-4c0b-4035-95c2-4bfa02a97d47 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:35.210571 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d99b0b4c-4c0b-4035-95c2-4bfa02a97d47 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:35.212582 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d99b0b4c-4c0b-4035-95c2-4bfa02a97d47 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:35.213073 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d99b0b4c-4c0b-4035-95c2-4bfa02a97d47 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:35.213502 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d99b0b4c-4c0b-4035-95c2-4bfa02a97d47 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:35.446188 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-d99b0b4c-4c0b-4035-95c2-4bfa02a97d47 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:35.461984 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-d99b0b4c-4c0b-4035-95c2-4bfa02a97d47 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1726 microversion: 2.1 time: 0.263030 Aug 08 02:27:35.478791 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 134/268] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:35 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1726 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:27:36.028977 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-c501c9cc-83bb-4348-bd86-6984ded2ac43 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:36.029719 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-c501c9cc-83bb-4348-bd86-6984ded2ac43 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:36.036693 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c501c9cc-83bb-4348-bd86-6984ded2ac43 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:36.037272 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c501c9cc-83bb-4348-bd86-6984ded2ac43 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:36.037618 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c501c9cc-83bb-4348-bd86-6984ded2ac43 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:36.038305 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c501c9cc-83bb-4348-bd86-6984ded2ac43 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:36.038753 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c501c9cc-83bb-4348-bd86-6984ded2ac43 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:36.039189 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c501c9cc-83bb-4348-bd86-6984ded2ac43 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:36.159147 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-7ae20299-a04f-4a02-a132-5a0502f2132c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:36.159752 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-7ae20299-a04f-4a02-a132-5a0502f2132c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:36.166145 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7ae20299-a04f-4a02-a132-5a0502f2132c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:36.166626 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7ae20299-a04f-4a02-a132-5a0502f2132c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:36.167059 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7ae20299-a04f-4a02-a132-5a0502f2132c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:36.167748 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7ae20299-a04f-4a02-a132-5a0502f2132c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:36.168194 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7ae20299-a04f-4a02-a132-5a0502f2132c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:36.168611 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7ae20299-a04f-4a02-a132-5a0502f2132c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:36.282584 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-c501c9cc-83bb-4348-bd86-6984ded2ac43 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:36.293161 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-c501c9cc-83bb-4348-bd86-6984ded2ac43 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:36.295205 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-c501c9cc-83bb-4348-bd86-6984ded2ac43 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:36.297711 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-c501c9cc-83bb-4348-bd86-6984ded2ac43 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1725 microversion: 2.20 time: 0.272516 Aug 08 02:27:36.310495 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 135/269] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:27:36 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1725 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:27:36.455359 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-7ae20299-a04f-4a02-a132-5a0502f2132c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:36.461658 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-7ae20299-a04f-4a02-a132-5a0502f2132c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1372 microversion: 2.1 time: 0.306492 Aug 08 02:27:36.462228 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 135/270] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:36 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1372 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:27:36.492511 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-cd54b0f2-0335-49e3-93e4-cd6985773a56 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:36.493264 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-cd54b0f2-0335-49e3-93e4-cd6985773a56 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:36.501117 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cd54b0f2-0335-49e3-93e4-cd6985773a56 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:36.501815 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cd54b0f2-0335-49e3-93e4-cd6985773a56 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:36.502796 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cd54b0f2-0335-49e3-93e4-cd6985773a56 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:36.503765 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cd54b0f2-0335-49e3-93e4-cd6985773a56 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:36.504281 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cd54b0f2-0335-49e3-93e4-cd6985773a56 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:36.504791 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cd54b0f2-0335-49e3-93e4-cd6985773a56 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:36.713455 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-cd54b0f2-0335-49e3-93e4-cd6985773a56 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:36.722424 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-cd54b0f2-0335-49e3-93e4-cd6985773a56 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1726 microversion: 2.1 time: 0.236525 Aug 08 02:27:36.722991 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 136/271] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:36 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1726 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:36.891128 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-06c48965-aa6c-4cfc-9aab-86f232b75fe9 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1944311372", "imageRef": "30833f47-5987-41ca-b5ee-ad5ecafbb265", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "f92c936a-6950-4530-8001-5260ea91adac"}], "security_groups": [{"name": "tempest-securitygroup--1986282352"}], "key_name": "tempest-keypair-458871973", "user_data": "}} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:27:36.999799 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.network.neutron [None req-06c48965-aa6c-4cfc-9aab-86f232b75fe9 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] validate_networks() for [('f92c936a-6950-4530-8001-5260ea91adac', None, None, None, None, None)] {{(pid=118351) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 08 02:27:37.333256 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-ab4d3474-e2f3-4eac-9c1f-18f506a3c2ea tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:37.341444 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-ab4d3474-e2f3-4eac-9c1f-18f506a3c2ea tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:37.353593 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ab4d3474-e2f3-4eac-9c1f-18f506a3c2ea tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:37.353593 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ab4d3474-e2f3-4eac-9c1f-18f506a3c2ea tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:37.353593 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ab4d3474-e2f3-4eac-9c1f-18f506a3c2ea tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:37.353593 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ab4d3474-e2f3-4eac-9c1f-18f506a3c2ea tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:37.353922 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ab4d3474-e2f3-4eac-9c1f-18f506a3c2ea tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:37.353922 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ab4d3474-e2f3-4eac-9c1f-18f506a3c2ea tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:37.450067 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.virt.hardware [None req-06c48965-aa6c-4cfc-9aab-86f232b75fe9 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Flavor limits 0:0:0 {{(pid=118351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 08 02:27:37.451050 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.virt.hardware [None req-06c48965-aa6c-4cfc-9aab-86f232b75fe9 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Image limits 0:0:0 {{(pid=118351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 08 02:27:37.451529 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.virt.hardware [None req-06c48965-aa6c-4cfc-9aab-86f232b75fe9 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Flavor pref 0:0:0 {{(pid=118351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 08 02:27:37.451957 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.virt.hardware [None req-06c48965-aa6c-4cfc-9aab-86f232b75fe9 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Image pref 0:0:0 {{(pid=118351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 08 02:27:37.452960 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.virt.hardware [None req-06c48965-aa6c-4cfc-9aab-86f232b75fe9 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 08 02:27:37.461777 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-06c48965-aa6c-4cfc-9aab-86f232b75fe9 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:37.462437 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-06c48965-aa6c-4cfc-9aab-86f232b75fe9 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:37.462871 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-06c48965-aa6c-4cfc-9aab-86f232b75fe9 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:37.463604 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-06c48965-aa6c-4cfc-9aab-86f232b75fe9 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:37.464182 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-06c48965-aa6c-4cfc-9aab-86f232b75fe9 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:37.464557 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-06c48965-aa6c-4cfc-9aab-86f232b75fe9 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:37.482822 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.quota [None req-06c48965-aa6c-4cfc-9aab-86f232b75fe9 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Getting quotas for project ca29b2b9a59d4826bfd25f4dcfa9cffe. Resources: {'ram', 'cores', 'instances'} {{(pid=118351) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 08 02:27:37.486543 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.quota [None req-06c48965-aa6c-4cfc-9aab-86f232b75fe9 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Getting quotas for user 578cd837d9fe442587f8c6c8528334c0 and project ca29b2b9a59d4826bfd25f4dcfa9cffe. Resources: {'ram', 'cores', 'instances'} {{(pid=118351) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 08 02:27:37.500265 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-06c48965-aa6c-4cfc-9aab-86f232b75fe9 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Going to run 1 instances... {{(pid=118351) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Aug 08 02:27:37.502630 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-06c48965-aa6c-4cfc-9aab-86f232b75fe9 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: dda0c0b0-ed8e-4d43-9b26-abe1bca05a61] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='30833f47-5987-41ca-b5ee-ad5ecafbb265',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118351) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Aug 08 02:27:37.528201 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-ab4d3474-e2f3-4eac-9c1f-18f506a3c2ea tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:37.539691 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-ab4d3474-e2f3-4eac-9c1f-18f506a3c2ea tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:37.541271 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-ab4d3474-e2f3-4eac-9c1f-18f506a3c2ea tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:37.545789 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-ab4d3474-e2f3-4eac-9c1f-18f506a3c2ea tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1725 microversion: 2.20 time: 0.215266 Aug 08 02:27:37.545789 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 137/272] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:27:37 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1725 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 08 02:27:37.547735 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-de759632-76d7-4859-ad1f-c0ff16c431cb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:37.548347 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-de759632-76d7-4859-ad1f-c0ff16c431cb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:37.561003 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-de759632-76d7-4859-ad1f-c0ff16c431cb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:37.562044 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-de759632-76d7-4859-ad1f-c0ff16c431cb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:37.562494 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-de759632-76d7-4859-ad1f-c0ff16c431cb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:37.563409 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-de759632-76d7-4859-ad1f-c0ff16c431cb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:37.564130 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-de759632-76d7-4859-ad1f-c0ff16c431cb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:37.564554 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-de759632-76d7-4859-ad1f-c0ff16c431cb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:37.626598 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-06c48965-aa6c-4cfc-9aab-86f232b75fe9 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.743088 Aug 08 02:27:37.626598 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 136/273] 10.222.0.27 () {60 vars in 1147 bytes} [Mon Aug 8 02:27:36 2022] POST /compute/v2.1/servers => generated 402 bytes in 745 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 08 02:27:37.633479 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [req-076761ea-86e0-4f7b-9692-30d82cd38122 req-43ffc981-439c-4c12-9365-16b39f92cab7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a5b1155c-5bba-46ae-8f3c-61f70c8a8a24", "tag": "9b3bb00c-894f-4b64-935c-89a64519fd90"}]} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:27:37.646330 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-076761ea-86e0-4f7b-9692-30d82cd38122 req-43ffc981-439c-4c12-9365-16b39f92cab7 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:37.647013 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-076761ea-86e0-4f7b-9692-30d82cd38122 req-43ffc981-439c-4c12-9365-16b39f92cab7 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:37.647545 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-076761ea-86e0-4f7b-9692-30d82cd38122 req-43ffc981-439c-4c12-9365-16b39f92cab7 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:37.662265 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.compute.server_external_events [req-076761ea-86e0-4f7b-9692-30d82cd38122 req-43ffc981-439c-4c12-9365-16b39f92cab7 service nova] Creating event network-changed:9b3bb00c-894f-4b64-935c-89a64519fd90 for instance a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 on hv-cinder-74014 Aug 08 02:27:37.667716 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [req-076761ea-86e0-4f7b-9692-30d82cd38122 req-43ffc981-439c-4c12-9365-16b39f92cab7 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039296 Aug 08 02:27:37.668577 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 137/274] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:27:37 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:27:37.675738 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-d47d0873-3ecd-4d6d-841d-024d3bad1545 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:37.677126 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-d47d0873-3ecd-4d6d-841d-024d3bad1545 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: dda0c0b0-ed8e-4d43-9b26-abe1bca05a61] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:37.750093 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-de759632-76d7-4859-ad1f-c0ff16c431cb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:37.770362 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-de759632-76d7-4859-ad1f-c0ff16c431cb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1372 microversion: 2.1 time: 0.226113 Aug 08 02:27:37.771107 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 138/275] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:37 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1372 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:37.774135 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-d47d0873-3ecd-4d6d-841d-024d3bad1545 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:37.775823 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-f3a104f8-76cb-4504-a972-2c4381d80d1a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:37.776477 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-f3a104f8-76cb-4504-a972-2c4381d80d1a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:37.783367 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f3a104f8-76cb-4504-a972-2c4381d80d1a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:37.784005 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f3a104f8-76cb-4504-a972-2c4381d80d1a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:37.784641 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f3a104f8-76cb-4504-a972-2c4381d80d1a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:37.785473 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f3a104f8-76cb-4504-a972-2c4381d80d1a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:37.786021 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f3a104f8-76cb-4504-a972-2c4381d80d1a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:37.786756 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f3a104f8-76cb-4504-a972-2c4381d80d1a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:37.799777 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-d47d0873-3ecd-4d6d-841d-024d3bad1545 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61" status: 200 len: 1241 microversion: 2.1 time: 0.129506 Aug 08 02:27:37.800363 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 138/276] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:37 2022] GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 => generated 1241 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:38.165515 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-f3a104f8-76cb-4504-a972-2c4381d80d1a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:38.169248 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:38.175654 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-f3a104f8-76cb-4504-a972-2c4381d80d1a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1726 microversion: 2.1 time: 0.403496 Aug 08 02:27:38.176917 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 139/277] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:37 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1726 bytes in 405 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:38.566539 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-746a0759-6d4e-426c-90fe-0f852246fa6e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:38.567586 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-746a0759-6d4e-426c-90fe-0f852246fa6e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:38.575746 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-746a0759-6d4e-426c-90fe-0f852246fa6e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:38.576222 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-746a0759-6d4e-426c-90fe-0f852246fa6e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:38.576658 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-746a0759-6d4e-426c-90fe-0f852246fa6e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:38.578405 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-746a0759-6d4e-426c-90fe-0f852246fa6e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:38.578405 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-746a0759-6d4e-426c-90fe-0f852246fa6e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:38.578635 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-746a0759-6d4e-426c-90fe-0f852246fa6e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:38.715178 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-746a0759-6d4e-426c-90fe-0f852246fa6e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:38.723840 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-746a0759-6d4e-426c-90fe-0f852246fa6e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:38.725385 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-746a0759-6d4e-426c-90fe-0f852246fa6e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:38.726538 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-746a0759-6d4e-426c-90fe-0f852246fa6e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1725 microversion: 2.20 time: 0.165699 Aug 08 02:27:38.727091 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 139/278] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:27:38 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1725 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 08 02:27:38.798165 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-daeece37-e32d-4cd6-b4cd-aad600db3260 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:38.799296 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-daeece37-e32d-4cd6-b4cd-aad600db3260 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:38.806198 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-daeece37-e32d-4cd6-b4cd-aad600db3260 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:38.806692 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-daeece37-e32d-4cd6-b4cd-aad600db3260 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:38.807129 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-daeece37-e32d-4cd6-b4cd-aad600db3260 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:38.807818 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-daeece37-e32d-4cd6-b4cd-aad600db3260 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:38.808271 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-daeece37-e32d-4cd6-b4cd-aad600db3260 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:38.808723 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-daeece37-e32d-4cd6-b4cd-aad600db3260 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:38.822082 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-f57a8d6d-b488-4ce3-a2c5-af942e897fbb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:38.822824 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-f57a8d6d-b488-4ce3-a2c5-af942e897fbb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: dda0c0b0-ed8e-4d43-9b26-abe1bca05a61] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:38.829430 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f57a8d6d-b488-4ce3-a2c5-af942e897fbb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:38.830044 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f57a8d6d-b488-4ce3-a2c5-af942e897fbb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:38.830552 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f57a8d6d-b488-4ce3-a2c5-af942e897fbb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:38.831250 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f57a8d6d-b488-4ce3-a2c5-af942e897fbb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:38.831705 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f57a8d6d-b488-4ce3-a2c5-af942e897fbb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:38.832130 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f57a8d6d-b488-4ce3-a2c5-af942e897fbb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:38.913135 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-f57a8d6d-b488-4ce3-a2c5-af942e897fbb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:38.922051 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-f57a8d6d-b488-4ce3-a2c5-af942e897fbb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61" status: 200 len: 1245 microversion: 2.1 time: 0.103038 Aug 08 02:27:38.926394 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 140/279] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:38 2022] GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 => generated 1245 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:38.978003 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-daeece37-e32d-4cd6-b4cd-aad600db3260 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:38.986167 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-daeece37-e32d-4cd6-b4cd-aad600db3260 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1372 microversion: 2.1 time: 0.192655 Aug 08 02:27:38.987079 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 140/280] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:38 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1372 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:39.197754 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-7357da69-25ea-4877-9104-b595bf97fa5c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:39.198287 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-7357da69-25ea-4877-9104-b595bf97fa5c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:39.204246 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7357da69-25ea-4877-9104-b595bf97fa5c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:39.204753 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7357da69-25ea-4877-9104-b595bf97fa5c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:39.205188 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7357da69-25ea-4877-9104-b595bf97fa5c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:39.205873 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7357da69-25ea-4877-9104-b595bf97fa5c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:39.206324 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7357da69-25ea-4877-9104-b595bf97fa5c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:39.206745 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7357da69-25ea-4877-9104-b595bf97fa5c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:39.335632 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-7357da69-25ea-4877-9104-b595bf97fa5c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:39.344129 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-7357da69-25ea-4877-9104-b595bf97fa5c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1726 microversion: 2.1 time: 0.150313 Aug 08 02:27:39.344717 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 141/281] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:39 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1726 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:39.750018 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-3aad7a09-9fad-41db-ac54-33998d793e1a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:39.750753 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-3aad7a09-9fad-41db-ac54-33998d793e1a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:39.756911 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3aad7a09-9fad-41db-ac54-33998d793e1a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:39.757445 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3aad7a09-9fad-41db-ac54-33998d793e1a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:39.757939 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3aad7a09-9fad-41db-ac54-33998d793e1a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:39.758646 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3aad7a09-9fad-41db-ac54-33998d793e1a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:39.759149 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3aad7a09-9fad-41db-ac54-33998d793e1a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:39.759601 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3aad7a09-9fad-41db-ac54-33998d793e1a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:39.911479 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-3aad7a09-9fad-41db-ac54-33998d793e1a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:39.919688 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-3aad7a09-9fad-41db-ac54-33998d793e1a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:39.921330 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-3aad7a09-9fad-41db-ac54-33998d793e1a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:39.922487 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-3aad7a09-9fad-41db-ac54-33998d793e1a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1725 microversion: 2.20 time: 0.176727 Aug 08 02:27:39.923055 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 141/282] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:27:39 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1725 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:27:39.945482 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-e1197f62-5b56-40a2-87b9-e1e6bf43f32b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:39.947543 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-e1197f62-5b56-40a2-87b9-e1e6bf43f32b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: dda0c0b0-ed8e-4d43-9b26-abe1bca05a61] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:39.954888 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e1197f62-5b56-40a2-87b9-e1e6bf43f32b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:39.960490 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e1197f62-5b56-40a2-87b9-e1e6bf43f32b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:39.960490 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e1197f62-5b56-40a2-87b9-e1e6bf43f32b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:39.960490 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e1197f62-5b56-40a2-87b9-e1e6bf43f32b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:39.960490 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e1197f62-5b56-40a2-87b9-e1e6bf43f32b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:39.961178 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e1197f62-5b56-40a2-87b9-e1e6bf43f32b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:40.010313 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-cb9e4d19-23e6-446b-bbc4-4e2ce4d10eca tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:40.016298 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-cb9e4d19-23e6-446b-bbc4-4e2ce4d10eca tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:40.018363 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cb9e4d19-23e6-446b-bbc4-4e2ce4d10eca tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:40.018888 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cb9e4d19-23e6-446b-bbc4-4e2ce4d10eca tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:40.020408 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cb9e4d19-23e6-446b-bbc4-4e2ce4d10eca tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:40.020408 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cb9e4d19-23e6-446b-bbc4-4e2ce4d10eca tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:40.020834 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cb9e4d19-23e6-446b-bbc4-4e2ce4d10eca tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:40.021368 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cb9e4d19-23e6-446b-bbc4-4e2ce4d10eca tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:40.031846 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-e1197f62-5b56-40a2-87b9-e1e6bf43f32b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:40.039875 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-e1197f62-5b56-40a2-87b9-e1e6bf43f32b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61" status: 200 len: 1245 microversion: 2.1 time: 0.098148 Aug 08 02:27:40.040451 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 142/283] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:39 2022] GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 => generated 1245 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:40.172568 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-cb9e4d19-23e6-446b-bbc4-4e2ce4d10eca tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:40.181558 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-cb9e4d19-23e6-446b-bbc4-4e2ce4d10eca tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1372 microversion: 2.1 time: 0.175167 Aug 08 02:27:40.182150 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 142/284] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:40 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1372 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:27:40.367299 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-d8ecbd3f-7dc0-4280-8825-d8d2ce7de7c6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:40.367779 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-d8ecbd3f-7dc0-4280-8825-d8d2ce7de7c6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:40.374356 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d8ecbd3f-7dc0-4280-8825-d8d2ce7de7c6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:40.374868 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d8ecbd3f-7dc0-4280-8825-d8d2ce7de7c6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:40.375309 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d8ecbd3f-7dc0-4280-8825-d8d2ce7de7c6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:40.376000 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d8ecbd3f-7dc0-4280-8825-d8d2ce7de7c6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:40.376456 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d8ecbd3f-7dc0-4280-8825-d8d2ce7de7c6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:40.376915 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d8ecbd3f-7dc0-4280-8825-d8d2ce7de7c6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:40.524191 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-d8ecbd3f-7dc0-4280-8825-d8d2ce7de7c6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:40.532777 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-d8ecbd3f-7dc0-4280-8825-d8d2ce7de7c6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1726 microversion: 2.1 time: 0.170056 Aug 08 02:27:40.533374 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 143/285] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:40 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1726 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:40.945373 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-ae9f5edb-b698-4c26-8fc1-5dc7fe8ea8d4 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:40.945989 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-ae9f5edb-b698-4c26-8fc1-5dc7fe8ea8d4 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:40.947841 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell0:SELECT=2 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:40.954720 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ae9f5edb-b698-4c26-8fc1-5dc7fe8ea8d4 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:40.955486 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ae9f5edb-b698-4c26-8fc1-5dc7fe8ea8d4 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:40.955928 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ae9f5edb-b698-4c26-8fc1-5dc7fe8ea8d4 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:40.956944 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ae9f5edb-b698-4c26-8fc1-5dc7fe8ea8d4 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:40.957404 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ae9f5edb-b698-4c26-8fc1-5dc7fe8ea8d4 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:40.962425 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ae9f5edb-b698-4c26-8fc1-5dc7fe8ea8d4 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:41.066974 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-2293a988-38cf-41c8-bc9e-e0c96002f42a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:41.067585 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-2293a988-38cf-41c8-bc9e-e0c96002f42a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: dda0c0b0-ed8e-4d43-9b26-abe1bca05a61] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:41.077248 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2293a988-38cf-41c8-bc9e-e0c96002f42a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:41.077749 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2293a988-38cf-41c8-bc9e-e0c96002f42a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:41.078313 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2293a988-38cf-41c8-bc9e-e0c96002f42a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:41.079023 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2293a988-38cf-41c8-bc9e-e0c96002f42a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:41.079519 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2293a988-38cf-41c8-bc9e-e0c96002f42a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:41.079856 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2293a988-38cf-41c8-bc9e-e0c96002f42a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:41.145354 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-ae9f5edb-b698-4c26-8fc1-5dc7fe8ea8d4 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:41.156621 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-ae9f5edb-b698-4c26-8fc1-5dc7fe8ea8d4 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:41.158436 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-ae9f5edb-b698-4c26-8fc1-5dc7fe8ea8d4 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:41.160060 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-ae9f5edb-b698-4c26-8fc1-5dc7fe8ea8d4 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1725 microversion: 2.20 time: 0.218441 Aug 08 02:27:41.160822 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 143/286] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:27:40 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1725 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 08 02:27:41.164924 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-2293a988-38cf-41c8-bc9e-e0c96002f42a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:41.176609 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-2293a988-38cf-41c8-bc9e-e0c96002f42a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61" status: 200 len: 1293 microversion: 2.1 time: 0.113516 Aug 08 02:27:41.178226 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 144/287] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:41 2022] GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 => generated 1293 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:41.205481 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-adfe671e-8de4-4118-aa04-0f6ad7d841ac tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:41.206113 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-adfe671e-8de4-4118-aa04-0f6ad7d841ac tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:41.212100 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-adfe671e-8de4-4118-aa04-0f6ad7d841ac tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:41.212582 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-adfe671e-8de4-4118-aa04-0f6ad7d841ac tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:41.213053 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-adfe671e-8de4-4118-aa04-0f6ad7d841ac tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:41.213738 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-adfe671e-8de4-4118-aa04-0f6ad7d841ac tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:41.214196 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-adfe671e-8de4-4118-aa04-0f6ad7d841ac tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:41.214620 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-adfe671e-8de4-4118-aa04-0f6ad7d841ac tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:41.369197 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-adfe671e-8de4-4118-aa04-0f6ad7d841ac tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:41.377748 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-adfe671e-8de4-4118-aa04-0f6ad7d841ac tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1372 microversion: 2.1 time: 0.176823 Aug 08 02:27:41.378323 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 144/288] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:41 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1372 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:41.554941 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-2237ffcf-d45d-44d0-814a-e6cb7822463f tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:41.555552 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-2237ffcf-d45d-44d0-814a-e6cb7822463f tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:41.561500 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2237ffcf-d45d-44d0-814a-e6cb7822463f tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:41.562575 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2237ffcf-d45d-44d0-814a-e6cb7822463f tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:41.563160 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2237ffcf-d45d-44d0-814a-e6cb7822463f tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:41.564099 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2237ffcf-d45d-44d0-814a-e6cb7822463f tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:41.564554 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2237ffcf-d45d-44d0-814a-e6cb7822463f tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:41.565052 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2237ffcf-d45d-44d0-814a-e6cb7822463f tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:41.711009 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-2237ffcf-d45d-44d0-814a-e6cb7822463f tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:41.721164 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-2237ffcf-d45d-44d0-814a-e6cb7822463f tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1726 microversion: 2.1 time: 0.169713 Aug 08 02:27:41.722139 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 145/289] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:41 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1726 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:42.185937 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-6f90cb3f-1aff-4d17-93af-078469709726 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:42.186796 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-6f90cb3f-1aff-4d17-93af-078469709726 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:42.193658 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6f90cb3f-1aff-4d17-93af-078469709726 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:42.194141 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6f90cb3f-1aff-4d17-93af-078469709726 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:42.194576 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6f90cb3f-1aff-4d17-93af-078469709726 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:42.195260 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6f90cb3f-1aff-4d17-93af-078469709726 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:42.195967 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6f90cb3f-1aff-4d17-93af-078469709726 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:42.196397 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6f90cb3f-1aff-4d17-93af-078469709726 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:42.200706 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-1d52a042-39e7-4c2f-8ffd-daa0c49a17fd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:42.201332 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-1d52a042-39e7-4c2f-8ffd-daa0c49a17fd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: dda0c0b0-ed8e-4d43-9b26-abe1bca05a61] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:42.208381 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1d52a042-39e7-4c2f-8ffd-daa0c49a17fd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:42.209246 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1d52a042-39e7-4c2f-8ffd-daa0c49a17fd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:42.209691 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1d52a042-39e7-4c2f-8ffd-daa0c49a17fd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:42.210391 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1d52a042-39e7-4c2f-8ffd-daa0c49a17fd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:42.210851 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1d52a042-39e7-4c2f-8ffd-daa0c49a17fd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:42.211274 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1d52a042-39e7-4c2f-8ffd-daa0c49a17fd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:42.291859 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-1d52a042-39e7-4c2f-8ffd-daa0c49a17fd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:42.299489 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-1d52a042-39e7-4c2f-8ffd-daa0c49a17fd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61" status: 200 len: 1293 microversion: 2.1 time: 0.102612 Aug 08 02:27:42.300294 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 146/290] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:42 2022] GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 => generated 1293 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:27:42.361913 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-6f90cb3f-1aff-4d17-93af-078469709726 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:42.369957 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-6f90cb3f-1aff-4d17-93af-078469709726 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:42.371536 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-6f90cb3f-1aff-4d17-93af-078469709726 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:42.372694 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-6f90cb3f-1aff-4d17-93af-078469709726 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1725 microversion: 2.20 time: 0.190639 Aug 08 02:27:42.373261 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 145/291] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:27:42 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1725 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:27:42.399327 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-0f9a8a47-3c58-4255-8e69-77057bb7f525 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:42.400248 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-0f9a8a47-3c58-4255-8e69-77057bb7f525 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:42.406487 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0f9a8a47-3c58-4255-8e69-77057bb7f525 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:42.406999 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0f9a8a47-3c58-4255-8e69-77057bb7f525 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:42.407491 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0f9a8a47-3c58-4255-8e69-77057bb7f525 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:42.408180 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0f9a8a47-3c58-4255-8e69-77057bb7f525 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:42.408636 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0f9a8a47-3c58-4255-8e69-77057bb7f525 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:42.409092 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0f9a8a47-3c58-4255-8e69-77057bb7f525 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:42.543732 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-0f9a8a47-3c58-4255-8e69-77057bb7f525 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:42.550500 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-0f9a8a47-3c58-4255-8e69-77057bb7f525 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1372 microversion: 2.1 time: 0.154998 Aug 08 02:27:42.551059 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 147/292] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:42 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1372 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:27:42.742642 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-fa86c13a-3b8b-4ca2-8a73-3786e65d1da6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:42.743273 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-fa86c13a-3b8b-4ca2-8a73-3786e65d1da6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:42.749283 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-fa86c13a-3b8b-4ca2-8a73-3786e65d1da6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:42.749847 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-fa86c13a-3b8b-4ca2-8a73-3786e65d1da6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:42.750290 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-fa86c13a-3b8b-4ca2-8a73-3786e65d1da6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:42.751002 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-fa86c13a-3b8b-4ca2-8a73-3786e65d1da6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:42.751459 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-fa86c13a-3b8b-4ca2-8a73-3786e65d1da6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:42.751884 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-fa86c13a-3b8b-4ca2-8a73-3786e65d1da6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:42.883238 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-fa86c13a-3b8b-4ca2-8a73-3786e65d1da6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:42.892196 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-fa86c13a-3b8b-4ca2-8a73-3786e65d1da6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1726 microversion: 2.1 time: 0.153354 Aug 08 02:27:42.892944 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 146/293] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:42 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1726 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:43.322030 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-d807a1fb-9e3d-4a96-8ee8-507d3c1a9b6c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:43.322862 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-d807a1fb-9e3d-4a96-8ee8-507d3c1a9b6c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: dda0c0b0-ed8e-4d43-9b26-abe1bca05a61] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:43.331118 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d807a1fb-9e3d-4a96-8ee8-507d3c1a9b6c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:43.331604 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d807a1fb-9e3d-4a96-8ee8-507d3c1a9b6c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:43.332040 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d807a1fb-9e3d-4a96-8ee8-507d3c1a9b6c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:43.332994 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d807a1fb-9e3d-4a96-8ee8-507d3c1a9b6c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:43.333567 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d807a1fb-9e3d-4a96-8ee8-507d3c1a9b6c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:43.334081 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d807a1fb-9e3d-4a96-8ee8-507d3c1a9b6c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:43.397339 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-ac9198ea-7e7b-49f4-b3a7-764844d98d5a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:43.398041 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-ac9198ea-7e7b-49f4-b3a7-764844d98d5a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:43.398190 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-d807a1fb-9e3d-4a96-8ee8-507d3c1a9b6c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:43.402431 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell1:SELECT=100 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:43.404888 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ac9198ea-7e7b-49f4-b3a7-764844d98d5a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:43.404888 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ac9198ea-7e7b-49f4-b3a7-764844d98d5a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:43.404888 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ac9198ea-7e7b-49f4-b3a7-764844d98d5a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:43.406212 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ac9198ea-7e7b-49f4-b3a7-764844d98d5a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:43.406212 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ac9198ea-7e7b-49f4-b3a7-764844d98d5a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:43.406612 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ac9198ea-7e7b-49f4-b3a7-764844d98d5a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:43.408344 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-d807a1fb-9e3d-4a96-8ee8-507d3c1a9b6c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61" status: 200 len: 1293 microversion: 2.1 time: 0.091088 Aug 08 02:27:43.409255 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 148/294] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:43 2022] GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 => generated 1293 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:43.540283 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-ac9198ea-7e7b-49f4-b3a7-764844d98d5a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:43.549815 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-ac9198ea-7e7b-49f4-b3a7-764844d98d5a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:43.551344 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-ac9198ea-7e7b-49f4-b3a7-764844d98d5a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:43.552412 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-ac9198ea-7e7b-49f4-b3a7-764844d98d5a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1725 microversion: 2.20 time: 0.160007 Aug 08 02:27:43.552991 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 147/295] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:27:43 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1725 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 08 02:27:43.572631 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-ae71c2de-f61a-4163-b1fb-bda10f37a618 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:43.573216 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-ae71c2de-f61a-4163-b1fb-bda10f37a618 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:43.579674 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ae71c2de-f61a-4163-b1fb-bda10f37a618 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:43.580154 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ae71c2de-f61a-4163-b1fb-bda10f37a618 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:43.580626 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ae71c2de-f61a-4163-b1fb-bda10f37a618 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:43.581348 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ae71c2de-f61a-4163-b1fb-bda10f37a618 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:43.581809 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ae71c2de-f61a-4163-b1fb-bda10f37a618 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:43.582261 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ae71c2de-f61a-4163-b1fb-bda10f37a618 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:43.713514 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-ae71c2de-f61a-4163-b1fb-bda10f37a618 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:43.721346 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-ae71c2de-f61a-4163-b1fb-bda10f37a618 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1372 microversion: 2.1 time: 0.153439 Aug 08 02:27:43.721902 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 149/296] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:43 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1372 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:43.913638 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-466d4c64-2316-4106-98aa-8a70085237fc tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:43.914284 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-466d4c64-2316-4106-98aa-8a70085237fc tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:43.916136 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell0:SELECT=10 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:43.916835 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_api:SELECT=84,nova_api:INSERT=4 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:43.922125 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-466d4c64-2316-4106-98aa-8a70085237fc tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:43.922616 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-466d4c64-2316-4106-98aa-8a70085237fc tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:43.923048 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-466d4c64-2316-4106-98aa-8a70085237fc tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:43.923738 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-466d4c64-2316-4106-98aa-8a70085237fc tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:43.924249 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-466d4c64-2316-4106-98aa-8a70085237fc tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:43.924686 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-466d4c64-2316-4106-98aa-8a70085237fc tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:44.063030 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-466d4c64-2316-4106-98aa-8a70085237fc tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:44.071775 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-466d4c64-2316-4106-98aa-8a70085237fc tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1726 microversion: 2.1 time: 0.161883 Aug 08 02:27:44.072380 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 148/297] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:43 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1726 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:44.430295 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-79331f4d-767f-4a11-ad2c-0a219bfd7f47 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:44.430982 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-79331f4d-767f-4a11-ad2c-0a219bfd7f47 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: dda0c0b0-ed8e-4d43-9b26-abe1bca05a61] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:44.433452 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=4 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:44.439423 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-79331f4d-767f-4a11-ad2c-0a219bfd7f47 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:44.439929 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-79331f4d-767f-4a11-ad2c-0a219bfd7f47 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:44.440360 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-79331f4d-767f-4a11-ad2c-0a219bfd7f47 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:44.441073 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-79331f4d-767f-4a11-ad2c-0a219bfd7f47 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:44.441524 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-79331f4d-767f-4a11-ad2c-0a219bfd7f47 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:44.441942 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-79331f4d-767f-4a11-ad2c-0a219bfd7f47 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:44.507307 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-79331f4d-767f-4a11-ad2c-0a219bfd7f47 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:44.515208 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-79331f4d-767f-4a11-ad2c-0a219bfd7f47 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61" status: 200 len: 1293 microversion: 2.1 time: 0.088391 Aug 08 02:27:44.516315 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 150/298] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:44 2022] GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 => generated 1293 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:44.578698 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-72cd1ef7-66cc-4b02-9d76-382c29b4ba80 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:44.579307 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-72cd1ef7-66cc-4b02-9d76-382c29b4ba80 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:44.585862 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-72cd1ef7-66cc-4b02-9d76-382c29b4ba80 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:44.586347 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-72cd1ef7-66cc-4b02-9d76-382c29b4ba80 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:44.586786 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-72cd1ef7-66cc-4b02-9d76-382c29b4ba80 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:44.587477 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-72cd1ef7-66cc-4b02-9d76-382c29b4ba80 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:44.587931 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-72cd1ef7-66cc-4b02-9d76-382c29b4ba80 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:44.588360 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-72cd1ef7-66cc-4b02-9d76-382c29b4ba80 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:44.729372 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-72cd1ef7-66cc-4b02-9d76-382c29b4ba80 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:44.739067 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-72cd1ef7-66cc-4b02-9d76-382c29b4ba80 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:44.740840 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-72cd1ef7-66cc-4b02-9d76-382c29b4ba80 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:44.741995 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-72cd1ef7-66cc-4b02-9d76-382c29b4ba80 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1725 microversion: 2.20 time: 0.168150 Aug 08 02:27:44.742623 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 149/299] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:27:44 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1725 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 08 02:27:44.742821 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-3f1a95f3-56ec-42cc-903e-c869f473a029 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:44.743431 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-3f1a95f3-56ec-42cc-903e-c869f473a029 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:44.750537 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3f1a95f3-56ec-42cc-903e-c869f473a029 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:44.751020 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3f1a95f3-56ec-42cc-903e-c869f473a029 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:44.751454 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3f1a95f3-56ec-42cc-903e-c869f473a029 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:44.752143 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3f1a95f3-56ec-42cc-903e-c869f473a029 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:44.752593 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3f1a95f3-56ec-42cc-903e-c869f473a029 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:44.753045 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3f1a95f3-56ec-42cc-903e-c869f473a029 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:44.892875 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-3f1a95f3-56ec-42cc-903e-c869f473a029 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:44.900838 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-3f1a95f3-56ec-42cc-903e-c869f473a029 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1372 microversion: 2.1 time: 0.162333 Aug 08 02:27:44.901395 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 151/300] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:44 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1372 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:27:45.092650 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-788ac075-cbab-4384-92c9-e57342c1c351 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:45.093573 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-788ac075-cbab-4384-92c9-e57342c1c351 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:45.099863 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-788ac075-cbab-4384-92c9-e57342c1c351 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:45.100558 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-788ac075-cbab-4384-92c9-e57342c1c351 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:45.101327 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-788ac075-cbab-4384-92c9-e57342c1c351 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:45.102243 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-788ac075-cbab-4384-92c9-e57342c1c351 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:45.103063 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-788ac075-cbab-4384-92c9-e57342c1c351 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:45.103681 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-788ac075-cbab-4384-92c9-e57342c1c351 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:45.241087 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-788ac075-cbab-4384-92c9-e57342c1c351 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:45.250979 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-788ac075-cbab-4384-92c9-e57342c1c351 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1726 microversion: 2.1 time: 0.161734 Aug 08 02:27:45.251589 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 150/301] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:45 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1726 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:45.535523 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-734e4ebc-b264-451b-a7aa-d6cfbef63067 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:45.536223 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-734e4ebc-b264-451b-a7aa-d6cfbef63067 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: dda0c0b0-ed8e-4d43-9b26-abe1bca05a61] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:45.542923 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-734e4ebc-b264-451b-a7aa-d6cfbef63067 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:45.543481 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-734e4ebc-b264-451b-a7aa-d6cfbef63067 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:45.543922 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-734e4ebc-b264-451b-a7aa-d6cfbef63067 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:45.544620 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-734e4ebc-b264-451b-a7aa-d6cfbef63067 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:45.545107 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-734e4ebc-b264-451b-a7aa-d6cfbef63067 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:45.545531 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-734e4ebc-b264-451b-a7aa-d6cfbef63067 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:45.612493 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-734e4ebc-b264-451b-a7aa-d6cfbef63067 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:45.618640 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-734e4ebc-b264-451b-a7aa-d6cfbef63067 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61" status: 200 len: 1293 microversion: 2.1 time: 0.087066 Aug 08 02:27:45.619189 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 152/302] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:45 2022] GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 => generated 1293 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:45.766732 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-8542f68e-247e-4378-bf44-cd6a8c073435 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:45.767477 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-8542f68e-247e-4378-bf44-cd6a8c073435 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:45.774986 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8542f68e-247e-4378-bf44-cd6a8c073435 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:45.775500 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8542f68e-247e-4378-bf44-cd6a8c073435 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:45.775965 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8542f68e-247e-4378-bf44-cd6a8c073435 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:45.776871 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8542f68e-247e-4378-bf44-cd6a8c073435 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:45.777332 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8542f68e-247e-4378-bf44-cd6a8c073435 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:45.777865 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8542f68e-247e-4378-bf44-cd6a8c073435 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:45.914536 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-8542f68e-247e-4378-bf44-cd6a8c073435 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:45.920129 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-5f9ab79a-09f6-4747-b993-099ca6565d72 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:45.922266 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-5f9ab79a-09f6-4747-b993-099ca6565d72 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:45.923632 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-8542f68e-247e-4378-bf44-cd6a8c073435 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:45.925582 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-8542f68e-247e-4378-bf44-cd6a8c073435 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:45.927144 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-8542f68e-247e-4378-bf44-cd6a8c073435 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1725 microversion: 2.20 time: 0.164900 Aug 08 02:27:45.927693 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 151/303] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:27:45 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1725 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:27:45.928155 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5f9ab79a-09f6-4747-b993-099ca6565d72 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:45.928621 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5f9ab79a-09f6-4747-b993-099ca6565d72 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:45.929133 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5f9ab79a-09f6-4747-b993-099ca6565d72 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:45.929823 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5f9ab79a-09f6-4747-b993-099ca6565d72 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:45.930296 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5f9ab79a-09f6-4747-b993-099ca6565d72 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:45.930851 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5f9ab79a-09f6-4747-b993-099ca6565d72 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:46.069984 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-5f9ab79a-09f6-4747-b993-099ca6565d72 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:46.078135 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-5f9ab79a-09f6-4747-b993-099ca6565d72 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1372 microversion: 2.1 time: 0.161777 Aug 08 02:27:46.084788 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 153/304] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:45 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1372 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:27:46.274995 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-9eb9f3ff-047e-48e6-a79e-9231a9cf827a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:46.275862 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-9eb9f3ff-047e-48e6-a79e-9231a9cf827a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:46.282806 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9eb9f3ff-047e-48e6-a79e-9231a9cf827a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:46.283300 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9eb9f3ff-047e-48e6-a79e-9231a9cf827a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:46.283737 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9eb9f3ff-047e-48e6-a79e-9231a9cf827a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:46.284590 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9eb9f3ff-047e-48e6-a79e-9231a9cf827a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:46.285185 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9eb9f3ff-047e-48e6-a79e-9231a9cf827a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:46.285505 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9eb9f3ff-047e-48e6-a79e-9231a9cf827a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:46.424000 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-9eb9f3ff-047e-48e6-a79e-9231a9cf827a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:46.434599 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-9eb9f3ff-047e-48e6-a79e-9231a9cf827a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1726 microversion: 2.1 time: 0.163539 Aug 08 02:27:46.435319 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 152/305] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:46 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1726 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:46.643439 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-6c4e628c-faf5-43d5-8dbb-bc5ccb5d098a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:46.643439 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-6c4e628c-faf5-43d5-8dbb-bc5ccb5d098a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: dda0c0b0-ed8e-4d43-9b26-abe1bca05a61] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:46.648150 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6c4e628c-faf5-43d5-8dbb-bc5ccb5d098a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:46.648792 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6c4e628c-faf5-43d5-8dbb-bc5ccb5d098a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:46.649351 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6c4e628c-faf5-43d5-8dbb-bc5ccb5d098a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:46.650269 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6c4e628c-faf5-43d5-8dbb-bc5ccb5d098a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:46.650857 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6c4e628c-faf5-43d5-8dbb-bc5ccb5d098a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:46.651414 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6c4e628c-faf5-43d5-8dbb-bc5ccb5d098a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:46.721183 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-6c4e628c-faf5-43d5-8dbb-bc5ccb5d098a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:46.729262 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-6c4e628c-faf5-43d5-8dbb-bc5ccb5d098a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61" status: 200 len: 1299 microversion: 2.1 time: 0.093167 Aug 08 02:27:46.729607 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 154/306] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:46 2022] GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 => generated 1299 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:46.952974 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-da63fa09-f028-40aa-9b37-dc7d6ac71258 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:46.952974 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-da63fa09-f028-40aa-9b37-dc7d6ac71258 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:46.959712 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-da63fa09-f028-40aa-9b37-dc7d6ac71258 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:46.960363 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-da63fa09-f028-40aa-9b37-dc7d6ac71258 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:46.960838 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-da63fa09-f028-40aa-9b37-dc7d6ac71258 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:46.961537 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-da63fa09-f028-40aa-9b37-dc7d6ac71258 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:46.961987 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-da63fa09-f028-40aa-9b37-dc7d6ac71258 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:46.962431 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-da63fa09-f028-40aa-9b37-dc7d6ac71258 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:47.101305 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-96527b90-272f-422a-a1ed-3cd3ae59cec2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:47.101918 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-96527b90-272f-422a-a1ed-3cd3ae59cec2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:47.109349 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-96527b90-272f-422a-a1ed-3cd3ae59cec2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:47.110074 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-96527b90-272f-422a-a1ed-3cd3ae59cec2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:47.110665 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-96527b90-272f-422a-a1ed-3cd3ae59cec2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:47.111683 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-96527b90-272f-422a-a1ed-3cd3ae59cec2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:47.112265 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-96527b90-272f-422a-a1ed-3cd3ae59cec2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:47.112891 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-96527b90-272f-422a-a1ed-3cd3ae59cec2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:47.168915 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-da63fa09-f028-40aa-9b37-dc7d6ac71258 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:47.180802 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-da63fa09-f028-40aa-9b37-dc7d6ac71258 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:47.184114 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-da63fa09-f028-40aa-9b37-dc7d6ac71258 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:47.186998 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-da63fa09-f028-40aa-9b37-dc7d6ac71258 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1725 microversion: 2.20 time: 0.239573 Aug 08 02:27:47.205789 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 153/307] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:27:46 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1725 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:27:47.289870 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-96527b90-272f-422a-a1ed-3cd3ae59cec2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:47.299870 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-96527b90-272f-422a-a1ed-3cd3ae59cec2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1372 microversion: 2.1 time: 0.203497 Aug 08 02:27:47.300952 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 155/308] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:47 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1372 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:47.376968 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [req-728602cb-c2ba-424b-84ae-aaa30ed72707 req-83b2df98-e1f1-416e-9de5-d5cf63e3058a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a5b1155c-5bba-46ae-8f3c-61f70c8a8a24", "name": "network-vif-plugged", "status": "completed", "tag": "9b3bb00c-894f-4b64-935c-89a64519fd90"}]} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:27:47.386252 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-728602cb-c2ba-424b-84ae-aaa30ed72707 req-83b2df98-e1f1-416e-9de5-d5cf63e3058a service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:47.386724 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-728602cb-c2ba-424b-84ae-aaa30ed72707 req-83b2df98-e1f1-416e-9de5-d5cf63e3058a service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:47.387321 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-728602cb-c2ba-424b-84ae-aaa30ed72707 req-83b2df98-e1f1-416e-9de5-d5cf63e3058a service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:47.398993 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.compute.server_external_events [req-728602cb-c2ba-424b-84ae-aaa30ed72707 req-83b2df98-e1f1-416e-9de5-d5cf63e3058a service nova] Creating event network-vif-plugged:9b3bb00c-894f-4b64-935c-89a64519fd90 for instance a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 on hv-cinder-74014 Aug 08 02:27:47.404162 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [req-728602cb-c2ba-424b-84ae-aaa30ed72707 req-83b2df98-e1f1-416e-9de5-d5cf63e3058a service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031636 Aug 08 02:27:47.404772 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 154/309] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:27:47 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:27:47.457955 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-b6ebecd5-5ff5-459b-917d-48501d3a8d13 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:47.458485 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-b6ebecd5-5ff5-459b-917d-48501d3a8d13 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:47.465173 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b6ebecd5-5ff5-459b-917d-48501d3a8d13 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:47.466856 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b6ebecd5-5ff5-459b-917d-48501d3a8d13 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:47.466856 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b6ebecd5-5ff5-459b-917d-48501d3a8d13 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:47.468039 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b6ebecd5-5ff5-459b-917d-48501d3a8d13 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:47.468698 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b6ebecd5-5ff5-459b-917d-48501d3a8d13 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:47.469464 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b6ebecd5-5ff5-459b-917d-48501d3a8d13 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:47.474735 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell0:SELECT=2 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:47.612361 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-b6ebecd5-5ff5-459b-917d-48501d3a8d13 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:47.620437 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-b6ebecd5-5ff5-459b-917d-48501d3a8d13 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1726 microversion: 2.1 time: 0.166853 Aug 08 02:27:47.621029 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 156/310] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:47 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1726 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:47.750043 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-a332be56-688c-4de5-ab9b-e96650eb93fc tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:47.751023 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-a332be56-688c-4de5-ab9b-e96650eb93fc tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: dda0c0b0-ed8e-4d43-9b26-abe1bca05a61] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:47.757328 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a332be56-688c-4de5-ab9b-e96650eb93fc tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:47.757808 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a332be56-688c-4de5-ab9b-e96650eb93fc tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:47.758401 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a332be56-688c-4de5-ab9b-e96650eb93fc tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:47.759271 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a332be56-688c-4de5-ab9b-e96650eb93fc tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:47.759860 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a332be56-688c-4de5-ab9b-e96650eb93fc tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:47.760466 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a332be56-688c-4de5-ab9b-e96650eb93fc tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:47.895631 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-a332be56-688c-4de5-ab9b-e96650eb93fc tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:47.903587 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-a332be56-688c-4de5-ab9b-e96650eb93fc tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61" status: 200 len: 1367 microversion: 2.1 time: 0.158241 Aug 08 02:27:47.911122 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 155/311] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:47 2022] GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 => generated 1367 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:48.003290 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [req-4f7e14d6-4fdf-47e3-9846-85da5a7c1ec1 req-d31f47f5-e1ce-4f32-8075-5b496a316b9c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "dda0c0b0-ed8e-4d43-9b26-abe1bca05a61", "tag": "c32805ad-aa79-4a4e-bcb8-fdaa7a625107"}]} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:27:48.009191 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell0:SELECT=14 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:48.014561 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-4f7e14d6-4fdf-47e3-9846-85da5a7c1ec1 req-d31f47f5-e1ce-4f32-8075-5b496a316b9c service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:48.015096 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-4f7e14d6-4fdf-47e3-9846-85da5a7c1ec1 req-d31f47f5-e1ce-4f32-8075-5b496a316b9c service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:48.015530 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-4f7e14d6-4fdf-47e3-9846-85da5a7c1ec1 req-d31f47f5-e1ce-4f32-8075-5b496a316b9c service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:48.028058 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.compute.server_external_events [req-4f7e14d6-4fdf-47e3-9846-85da5a7c1ec1 req-d31f47f5-e1ce-4f32-8075-5b496a316b9c service nova] Creating event network-changed:c32805ad-aa79-4a4e-bcb8-fdaa7a625107 for instance dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 on hv-cinder-74014 Aug 08 02:27:48.033945 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [req-4f7e14d6-4fdf-47e3-9846-85da5a7c1ec1 req-d31f47f5-e1ce-4f32-8075-5b496a316b9c service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034249 Aug 08 02:27:48.034546 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 157/312] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:27:47 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:27:48.215579 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-d9b0c180-17c8-43dd-a799-42abd7a186ef tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:48.215964 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-d9b0c180-17c8-43dd-a799-42abd7a186ef tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:48.223931 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d9b0c180-17c8-43dd-a799-42abd7a186ef tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:48.224422 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d9b0c180-17c8-43dd-a799-42abd7a186ef tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:48.224896 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d9b0c180-17c8-43dd-a799-42abd7a186ef tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:48.225599 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d9b0c180-17c8-43dd-a799-42abd7a186ef tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:48.226106 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d9b0c180-17c8-43dd-a799-42abd7a186ef tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:48.226532 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d9b0c180-17c8-43dd-a799-42abd7a186ef tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:48.325350 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-34651d31-993d-46ba-8313-1a77b0beca3f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:48.328270 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-34651d31-993d-46ba-8313-1a77b0beca3f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:48.332895 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-34651d31-993d-46ba-8313-1a77b0beca3f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:48.333597 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-34651d31-993d-46ba-8313-1a77b0beca3f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:48.334195 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-34651d31-993d-46ba-8313-1a77b0beca3f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:48.335065 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-34651d31-993d-46ba-8313-1a77b0beca3f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:48.335521 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-34651d31-993d-46ba-8313-1a77b0beca3f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:48.336132 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-34651d31-993d-46ba-8313-1a77b0beca3f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:48.366984 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-d9b0c180-17c8-43dd-a799-42abd7a186ef tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:48.377300 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-d9b0c180-17c8-43dd-a799-42abd7a186ef tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:48.382407 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-d9b0c180-17c8-43dd-a799-42abd7a186ef tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:48.383601 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-d9b0c180-17c8-43dd-a799-42abd7a186ef tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1725 microversion: 2.20 time: 0.172618 Aug 08 02:27:48.395499 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 156/313] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:27:48 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1725 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:27:48.515319 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-34651d31-993d-46ba-8313-1a77b0beca3f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:48.522988 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-34651d31-993d-46ba-8313-1a77b0beca3f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1372 microversion: 2.1 time: 0.204933 Aug 08 02:27:48.523495 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 158/314] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:48 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1372 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:27:48.642433 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-c823b36f-3a4d-48b6-bc5b-774bdf9da7e0 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:48.643054 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-c823b36f-3a4d-48b6-bc5b-774bdf9da7e0 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:48.650132 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c823b36f-3a4d-48b6-bc5b-774bdf9da7e0 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:48.650622 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c823b36f-3a4d-48b6-bc5b-774bdf9da7e0 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:48.651101 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c823b36f-3a4d-48b6-bc5b-774bdf9da7e0 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:48.651790 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c823b36f-3a4d-48b6-bc5b-774bdf9da7e0 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:48.652243 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c823b36f-3a4d-48b6-bc5b-774bdf9da7e0 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:48.652698 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c823b36f-3a4d-48b6-bc5b-774bdf9da7e0 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:48.658482 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell1:SELECT=100 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:48.801631 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-c823b36f-3a4d-48b6-bc5b-774bdf9da7e0 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:48.804554 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-c823b36f-3a4d-48b6-bc5b-774bdf9da7e0 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1726 microversion: 2.1 time: 0.166288 Aug 08 02:27:48.805171 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 157/315] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:48 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1726 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:48.925618 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-988d4fe0-fcc7-4099-8894-40e8fdb52c6a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:48.926236 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-988d4fe0-fcc7-4099-8894-40e8fdb52c6a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: dda0c0b0-ed8e-4d43-9b26-abe1bca05a61] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:48.932349 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-988d4fe0-fcc7-4099-8894-40e8fdb52c6a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:48.932866 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-988d4fe0-fcc7-4099-8894-40e8fdb52c6a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:48.933563 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-988d4fe0-fcc7-4099-8894-40e8fdb52c6a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:48.934296 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-988d4fe0-fcc7-4099-8894-40e8fdb52c6a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:48.934901 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-988d4fe0-fcc7-4099-8894-40e8fdb52c6a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:48.935321 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-988d4fe0-fcc7-4099-8894-40e8fdb52c6a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:49.089948 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-988d4fe0-fcc7-4099-8894-40e8fdb52c6a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:49.098102 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-988d4fe0-fcc7-4099-8894-40e8fdb52c6a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61" status: 200 len: 1367 microversion: 2.1 time: 0.176386 Aug 08 02:27:49.098678 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 159/316] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:48 2022] GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 => generated 1367 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:49.410927 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-83e89b8f-2958-49c6-8250-3837ac66d38e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:49.411648 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-83e89b8f-2958-49c6-8250-3837ac66d38e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:49.418208 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-83e89b8f-2958-49c6-8250-3837ac66d38e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:49.419908 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-83e89b8f-2958-49c6-8250-3837ac66d38e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:49.419908 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-83e89b8f-2958-49c6-8250-3837ac66d38e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:49.419908 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-83e89b8f-2958-49c6-8250-3837ac66d38e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:49.420398 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-83e89b8f-2958-49c6-8250-3837ac66d38e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:49.420875 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-83e89b8f-2958-49c6-8250-3837ac66d38e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:49.544079 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-6c98c974-87d5-4f1e-a177-f312bb1a03e3 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:49.544743 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-6c98c974-87d5-4f1e-a177-f312bb1a03e3 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:49.551757 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6c98c974-87d5-4f1e-a177-f312bb1a03e3 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:49.552177 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6c98c974-87d5-4f1e-a177-f312bb1a03e3 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:49.552611 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6c98c974-87d5-4f1e-a177-f312bb1a03e3 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:49.553321 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6c98c974-87d5-4f1e-a177-f312bb1a03e3 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:49.553771 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6c98c974-87d5-4f1e-a177-f312bb1a03e3 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:49.554225 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6c98c974-87d5-4f1e-a177-f312bb1a03e3 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:49.554603 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-83e89b8f-2958-49c6-8250-3837ac66d38e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:49.567738 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-83e89b8f-2958-49c6-8250-3837ac66d38e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:49.567738 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-83e89b8f-2958-49c6-8250-3837ac66d38e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:49.567738 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-83e89b8f-2958-49c6-8250-3837ac66d38e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1725 microversion: 2.20 time: 0.160069 Aug 08 02:27:49.568170 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 158/317] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:27:49 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1725 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:27:49.705824 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-6c98c974-87d5-4f1e-a177-f312bb1a03e3 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:49.714684 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-6c98c974-87d5-4f1e-a177-f312bb1a03e3 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1372 microversion: 2.1 time: 0.174369 Aug 08 02:27:49.715463 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 160/318] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:49 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1372 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:27:49.827512 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-7361c3e9-7a2b-4383-857c-e95621453e7e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:49.828119 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-7361c3e9-7a2b-4383-857c-e95621453e7e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:49.836064 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7361c3e9-7a2b-4383-857c-e95621453e7e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:49.837141 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7361c3e9-7a2b-4383-857c-e95621453e7e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:49.837658 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7361c3e9-7a2b-4383-857c-e95621453e7e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:49.838646 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7361c3e9-7a2b-4383-857c-e95621453e7e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:49.839277 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7361c3e9-7a2b-4383-857c-e95621453e7e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:49.839705 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7361c3e9-7a2b-4383-857c-e95621453e7e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:49.984320 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-7361c3e9-7a2b-4383-857c-e95621453e7e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:49.992701 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-7361c3e9-7a2b-4383-857c-e95621453e7e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1726 microversion: 2.1 time: 0.170530 Aug 08 02:27:49.993275 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 159/319] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:49 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1726 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:50.118413 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-8416584a-5e49-4546-9e4c-26a77cee3586 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:50.119063 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-8416584a-5e49-4546-9e4c-26a77cee3586 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: dda0c0b0-ed8e-4d43-9b26-abe1bca05a61] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:50.125606 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8416584a-5e49-4546-9e4c-26a77cee3586 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:50.126090 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8416584a-5e49-4546-9e4c-26a77cee3586 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:50.126530 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8416584a-5e49-4546-9e4c-26a77cee3586 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:50.127208 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8416584a-5e49-4546-9e4c-26a77cee3586 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:50.127656 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8416584a-5e49-4546-9e4c-26a77cee3586 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:50.128078 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8416584a-5e49-4546-9e4c-26a77cee3586 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:50.269252 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-8416584a-5e49-4546-9e4c-26a77cee3586 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:50.276681 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-8416584a-5e49-4546-9e4c-26a77cee3586 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61" status: 200 len: 1367 microversion: 2.1 time: 0.162155 Aug 08 02:27:50.277242 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 161/320] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:50 2022] GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 => generated 1367 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:50.589396 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-b5f301b5-8b9e-4ce7-ba6b-0337f0fcee7f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:50.589396 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-b5f301b5-8b9e-4ce7-ba6b-0337f0fcee7f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:50.596709 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b5f301b5-8b9e-4ce7-ba6b-0337f0fcee7f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:50.597199 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b5f301b5-8b9e-4ce7-ba6b-0337f0fcee7f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:50.597639 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b5f301b5-8b9e-4ce7-ba6b-0337f0fcee7f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:50.598332 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b5f301b5-8b9e-4ce7-ba6b-0337f0fcee7f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:50.598785 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b5f301b5-8b9e-4ce7-ba6b-0337f0fcee7f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:50.599207 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b5f301b5-8b9e-4ce7-ba6b-0337f0fcee7f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:50.737388 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-b5f301b5-8b9e-4ce7-ba6b-0337f0fcee7f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:50.740463 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-adf61f28-0583-4a78-b5f3-26c0658a9649 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:50.741435 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-adf61f28-0583-4a78-b5f3-26c0658a9649 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:50.747736 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-b5f301b5-8b9e-4ce7-ba6b-0337f0fcee7f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:50.748885 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-adf61f28-0583-4a78-b5f3-26c0658a9649 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:50.749688 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-b5f301b5-8b9e-4ce7-ba6b-0337f0fcee7f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:50.750094 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-adf61f28-0583-4a78-b5f3-26c0658a9649 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:50.750834 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-adf61f28-0583-4a78-b5f3-26c0658a9649 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:50.750950 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-b5f301b5-8b9e-4ce7-ba6b-0337f0fcee7f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1725 microversion: 2.20 time: 0.166078 Aug 08 02:27:50.751512 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 160/321] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:27:50 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1725 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:27:50.753319 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-adf61f28-0583-4a78-b5f3-26c0658a9649 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:50.753319 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-adf61f28-0583-4a78-b5f3-26c0658a9649 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:50.753319 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-adf61f28-0583-4a78-b5f3-26c0658a9649 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:50.885118 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-adf61f28-0583-4a78-b5f3-26c0658a9649 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:50.893334 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-adf61f28-0583-4a78-b5f3-26c0658a9649 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1372 microversion: 2.1 time: 0.161633 Aug 08 02:27:50.893890 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 162/322] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:50 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1372 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:27:51.014318 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-840f9ae7-c551-4902-94a0-af289add0699 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:51.015027 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-840f9ae7-c551-4902-94a0-af289add0699 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:51.020972 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-840f9ae7-c551-4902-94a0-af289add0699 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:51.021439 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-840f9ae7-c551-4902-94a0-af289add0699 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:51.021875 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-840f9ae7-c551-4902-94a0-af289add0699 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:51.022555 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-840f9ae7-c551-4902-94a0-af289add0699 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:51.023006 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-840f9ae7-c551-4902-94a0-af289add0699 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:51.023434 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-840f9ae7-c551-4902-94a0-af289add0699 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:51.157703 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-840f9ae7-c551-4902-94a0-af289add0699 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:51.165943 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-840f9ae7-c551-4902-94a0-af289add0699 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1726 microversion: 2.1 time: 0.155615 Aug 08 02:27:51.166437 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 161/323] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:51 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1726 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:51.298136 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-0ae62af7-6393-420f-8c4b-8f7b7b20722e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:51.298752 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-0ae62af7-6393-420f-8c4b-8f7b7b20722e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: dda0c0b0-ed8e-4d43-9b26-abe1bca05a61] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:51.305190 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0ae62af7-6393-420f-8c4b-8f7b7b20722e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:51.305676 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0ae62af7-6393-420f-8c4b-8f7b7b20722e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:51.306105 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0ae62af7-6393-420f-8c4b-8f7b7b20722e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:51.306799 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0ae62af7-6393-420f-8c4b-8f7b7b20722e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:51.307252 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0ae62af7-6393-420f-8c4b-8f7b7b20722e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:51.307672 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0ae62af7-6393-420f-8c4b-8f7b7b20722e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:51.486816 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-0ae62af7-6393-420f-8c4b-8f7b7b20722e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:51.494706 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-0ae62af7-6393-420f-8c4b-8f7b7b20722e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61" status: 200 len: 1367 microversion: 2.1 time: 0.200689 Aug 08 02:27:51.495286 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 163/324] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:51 2022] GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 => generated 1367 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:51.780063 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-4884510b-f668-4303-b52f-bcc4ce596279 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:51.781289 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-4884510b-f668-4303-b52f-bcc4ce596279 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:51.787853 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4884510b-f668-4303-b52f-bcc4ce596279 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:51.788336 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4884510b-f668-4303-b52f-bcc4ce596279 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:51.788807 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4884510b-f668-4303-b52f-bcc4ce596279 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:51.789502 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4884510b-f668-4303-b52f-bcc4ce596279 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:51.789952 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4884510b-f668-4303-b52f-bcc4ce596279 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:51.790374 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4884510b-f668-4303-b52f-bcc4ce596279 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:51.917236 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-720ac0a4-d4e3-415c-bfbe-e3407b86543b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:51.918211 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-720ac0a4-d4e3-415c-bfbe-e3407b86543b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:51.927236 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-720ac0a4-d4e3-415c-bfbe-e3407b86543b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:51.927724 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-720ac0a4-d4e3-415c-bfbe-e3407b86543b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:51.928156 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-720ac0a4-d4e3-415c-bfbe-e3407b86543b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:51.928750 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-4884510b-f668-4303-b52f-bcc4ce596279 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:51.928874 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-720ac0a4-d4e3-415c-bfbe-e3407b86543b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:51.929316 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-720ac0a4-d4e3-415c-bfbe-e3407b86543b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:51.929741 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-720ac0a4-d4e3-415c-bfbe-e3407b86543b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:51.940366 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-4884510b-f668-4303-b52f-bcc4ce596279 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:51.940366 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-4884510b-f668-4303-b52f-bcc4ce596279 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:51.940941 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-4884510b-f668-4303-b52f-bcc4ce596279 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1725 microversion: 2.20 time: 0.164879 Aug 08 02:27:51.941679 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 162/325] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:27:51 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1725 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:27:52.069314 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-720ac0a4-d4e3-415c-bfbe-e3407b86543b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:52.076913 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-720ac0a4-d4e3-415c-bfbe-e3407b86543b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1372 microversion: 2.1 time: 0.163508 Aug 08 02:27:52.077938 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 164/326] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:51 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1372 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:52.187804 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-868f58e3-cd95-4bc4-8c4e-0be1f591640e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:52.188414 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-868f58e3-cd95-4bc4-8c4e-0be1f591640e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:52.194845 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-868f58e3-cd95-4bc4-8c4e-0be1f591640e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:52.195326 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-868f58e3-cd95-4bc4-8c4e-0be1f591640e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:52.195755 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-868f58e3-cd95-4bc4-8c4e-0be1f591640e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:52.196439 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-868f58e3-cd95-4bc4-8c4e-0be1f591640e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:52.197216 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-868f58e3-cd95-4bc4-8c4e-0be1f591640e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:52.197638 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-868f58e3-cd95-4bc4-8c4e-0be1f591640e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:52.332883 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-868f58e3-cd95-4bc4-8c4e-0be1f591640e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:52.341069 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-868f58e3-cd95-4bc4-8c4e-0be1f591640e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1726 microversion: 2.1 time: 0.157761 Aug 08 02:27:52.341638 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 163/327] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:52 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1726 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:27:52.515584 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-39cb30e0-29d7-489b-9ede-10bfc544e57c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:52.516302 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-39cb30e0-29d7-489b-9ede-10bfc544e57c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: dda0c0b0-ed8e-4d43-9b26-abe1bca05a61] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:52.522348 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-39cb30e0-29d7-489b-9ede-10bfc544e57c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:52.522820 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-39cb30e0-29d7-489b-9ede-10bfc544e57c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:52.523258 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-39cb30e0-29d7-489b-9ede-10bfc544e57c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:52.523938 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-39cb30e0-29d7-489b-9ede-10bfc544e57c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:52.524384 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-39cb30e0-29d7-489b-9ede-10bfc544e57c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:52.524833 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-39cb30e0-29d7-489b-9ede-10bfc544e57c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:52.659924 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-39cb30e0-29d7-489b-9ede-10bfc544e57c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:52.680022 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-39cb30e0-29d7-489b-9ede-10bfc544e57c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61" status: 200 len: 1367 microversion: 2.1 time: 0.168291 Aug 08 02:27:52.680577 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 165/328] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:52 2022] GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 => generated 1367 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:52.963415 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-4bc0faff-37f0-455a-af8c-fcfbf7b3c847 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:52.964956 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-4bc0faff-37f0-455a-af8c-fcfbf7b3c847 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:52.975448 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4bc0faff-37f0-455a-af8c-fcfbf7b3c847 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:52.976032 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4bc0faff-37f0-455a-af8c-fcfbf7b3c847 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:52.976713 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4bc0faff-37f0-455a-af8c-fcfbf7b3c847 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:52.977395 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4bc0faff-37f0-455a-af8c-fcfbf7b3c847 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:52.977854 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4bc0faff-37f0-455a-af8c-fcfbf7b3c847 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:52.978285 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4bc0faff-37f0-455a-af8c-fcfbf7b3c847 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:53.100351 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-88aebca9-2862-4820-bbb9-8b07ef861a0c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:53.101224 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-88aebca9-2862-4820-bbb9-8b07ef861a0c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:53.107627 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-88aebca9-2862-4820-bbb9-8b07ef861a0c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:53.108188 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-88aebca9-2862-4820-bbb9-8b07ef861a0c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:53.109107 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-88aebca9-2862-4820-bbb9-8b07ef861a0c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:53.109848 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-88aebca9-2862-4820-bbb9-8b07ef861a0c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:53.110316 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-88aebca9-2862-4820-bbb9-8b07ef861a0c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:53.110784 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-88aebca9-2862-4820-bbb9-8b07ef861a0c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:53.115094 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell1:SELECT=100 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:53.134516 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-4bc0faff-37f0-455a-af8c-fcfbf7b3c847 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:53.144397 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-4bc0faff-37f0-455a-af8c-fcfbf7b3c847 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:53.149335 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-4bc0faff-37f0-455a-af8c-fcfbf7b3c847 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:53.149335 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-4bc0faff-37f0-455a-af8c-fcfbf7b3c847 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1725 microversion: 2.20 time: 0.188001 Aug 08 02:27:53.149335 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 164/329] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:27:52 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1725 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:27:53.246955 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-88aebca9-2862-4820-bbb9-8b07ef861a0c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:53.254979 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-88aebca9-2862-4820-bbb9-8b07ef861a0c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1565 microversion: 2.1 time: 0.160083 Aug 08 02:27:53.255533 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 166/330] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:53 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1565 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:53.278139 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-ef252911-d092-48e5-ac31-bbb4a9a72413 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:53.278260 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-ef252911-d092-48e5-ac31-bbb4a9a72413 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:53.284626 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ef252911-d092-48e5-ac31-bbb4a9a72413 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:53.285132 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ef252911-d092-48e5-ac31-bbb4a9a72413 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:53.285559 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ef252911-d092-48e5-ac31-bbb4a9a72413 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:53.286249 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ef252911-d092-48e5-ac31-bbb4a9a72413 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:53.286695 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ef252911-d092-48e5-ac31-bbb4a9a72413 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:53.287115 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ef252911-d092-48e5-ac31-bbb4a9a72413 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:53.373188 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-ef252911-d092-48e5-ac31-bbb4a9a72413 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.099013 Aug 08 02:27:53.374025 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-1c8fb47c-2215-4fbf-a82c-d43e9ec706be tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:53.374712 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 165/331] 10.222.0.27 () {58 vars in 1275 bytes} [Mon Aug 8 02:27:53 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24/os-interface => generated 284 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:27:53.382979 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-1c8fb47c-2215-4fbf-a82c-d43e9ec706be tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:53.390731 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1c8fb47c-2215-4fbf-a82c-d43e9ec706be tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:53.391695 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1c8fb47c-2215-4fbf-a82c-d43e9ec706be tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:53.392513 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1c8fb47c-2215-4fbf-a82c-d43e9ec706be tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:53.393462 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1c8fb47c-2215-4fbf-a82c-d43e9ec706be tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:53.394077 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1c8fb47c-2215-4fbf-a82c-d43e9ec706be tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:53.394535 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1c8fb47c-2215-4fbf-a82c-d43e9ec706be tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:53.545493 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-1c8fb47c-2215-4fbf-a82c-d43e9ec706be tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:53.554201 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-1c8fb47c-2215-4fbf-a82c-d43e9ec706be tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1703 microversion: 2.1 time: 0.184047 Aug 08 02:27:53.554765 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 167/332] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:53 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1703 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:53.575489 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-960edda1-907b-49cb-8a59-48732179379b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:27:53.585758 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-960edda1-907b-49cb-8a59-48732179379b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:53.586504 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-960edda1-907b-49cb-8a59-48732179379b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:53.587095 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-960edda1-907b-49cb-8a59-48732179379b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:53.625175 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-960edda1-907b-49cb-8a59-48732179379b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Going to try to start instance {{(pid=118352) start /opt/stack/nova/nova/compute/api.py:2732}} Aug 08 02:27:53.701265 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-ea88d77d-2add-4b32-b8d3-0ad1159ce1ae tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:53.701896 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-ea88d77d-2add-4b32-b8d3-0ad1159ce1ae tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: dda0c0b0-ed8e-4d43-9b26-abe1bca05a61] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:53.708179 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ea88d77d-2add-4b32-b8d3-0ad1159ce1ae tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:53.708921 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ea88d77d-2add-4b32-b8d3-0ad1159ce1ae tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:53.709558 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ea88d77d-2add-4b32-b8d3-0ad1159ce1ae tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:53.710261 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ea88d77d-2add-4b32-b8d3-0ad1159ce1ae tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:53.710732 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ea88d77d-2add-4b32-b8d3-0ad1159ce1ae tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:53.711307 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ea88d77d-2add-4b32-b8d3-0ad1159ce1ae tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:53.853731 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-ea88d77d-2add-4b32-b8d3-0ad1159ce1ae tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:53.862163 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-ea88d77d-2add-4b32-b8d3-0ad1159ce1ae tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61" status: 200 len: 1367 microversion: 2.1 time: 0.164841 Aug 08 02:27:53.862664 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 168/333] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:53 2022] GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 => generated 1367 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:27:53.954518 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-960edda1-907b-49cb-8a59-48732179379b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "POST /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a/action" status: 202 len: 0 microversion: 2.1 time: 0.382503 Aug 08 02:27:53.955141 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 166/334] 10.222.0.27 () {60 vars in 1278 bytes} [Mon Aug 8 02:27:53 2022] POST /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a/action => generated 0 bytes in 384 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 08 02:27:53.963636 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-2fe4d454-29fb-4903-bebb-1f8dadbaa695 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:53.964337 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-2fe4d454-29fb-4903-bebb-1f8dadbaa695 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:53.971367 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2fe4d454-29fb-4903-bebb-1f8dadbaa695 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:53.971858 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2fe4d454-29fb-4903-bebb-1f8dadbaa695 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:53.972298 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2fe4d454-29fb-4903-bebb-1f8dadbaa695 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:53.973065 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2fe4d454-29fb-4903-bebb-1f8dadbaa695 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:53.973529 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2fe4d454-29fb-4903-bebb-1f8dadbaa695 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:53.973957 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2fe4d454-29fb-4903-bebb-1f8dadbaa695 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:54.122785 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-2fe4d454-29fb-4903-bebb-1f8dadbaa695 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:54.131256 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-2fe4d454-29fb-4903-bebb-1f8dadbaa695 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1712 microversion: 2.1 time: 0.171711 Aug 08 02:27:54.131905 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 169/335] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:53 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1712 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:54.172233 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-bdfa0964-db89-47fa-9ff3-ed34c435814f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:54.173025 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-bdfa0964-db89-47fa-9ff3-ed34c435814f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:54.181628 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-bdfa0964-db89-47fa-9ff3-ed34c435814f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:54.182492 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-bdfa0964-db89-47fa-9ff3-ed34c435814f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:54.183087 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-bdfa0964-db89-47fa-9ff3-ed34c435814f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:54.183949 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-bdfa0964-db89-47fa-9ff3-ed34c435814f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:54.184561 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-bdfa0964-db89-47fa-9ff3-ed34c435814f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:54.185018 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-bdfa0964-db89-47fa-9ff3-ed34c435814f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:54.339674 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-bdfa0964-db89-47fa-9ff3-ed34c435814f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:54.349895 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-bdfa0964-db89-47fa-9ff3-ed34c435814f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:54.351443 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-bdfa0964-db89-47fa-9ff3-ed34c435814f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:54.352599 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-bdfa0964-db89-47fa-9ff3-ed34c435814f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1725 microversion: 2.20 time: 0.185447 Aug 08 02:27:54.353532 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 167/336] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:27:54 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1725 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:27:54.883186 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-06e57ac3-b323-4919-9fbd-4ce1e0dd1b1a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:54.883808 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-06e57ac3-b323-4919-9fbd-4ce1e0dd1b1a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: dda0c0b0-ed8e-4d43-9b26-abe1bca05a61] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:54.889671 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-06e57ac3-b323-4919-9fbd-4ce1e0dd1b1a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:54.890204 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-06e57ac3-b323-4919-9fbd-4ce1e0dd1b1a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:54.901011 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-06e57ac3-b323-4919-9fbd-4ce1e0dd1b1a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:54.901011 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-06e57ac3-b323-4919-9fbd-4ce1e0dd1b1a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:54.901011 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-06e57ac3-b323-4919-9fbd-4ce1e0dd1b1a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:54.901011 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-06e57ac3-b323-4919-9fbd-4ce1e0dd1b1a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:54.987191 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [req-5c35d39c-ff3c-4ff4-9c55-459927ecbb5d req-ded693ff-72db-410d-91f0-f4a0ea4d0f7e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a5b1155c-5bba-46ae-8f3c-61f70c8a8a24", "tag": "9b3bb00c-894f-4b64-935c-89a64519fd90"}]} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:27:55.011904 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-5c35d39c-ff3c-4ff4-9c55-459927ecbb5d req-ded693ff-72db-410d-91f0-f4a0ea4d0f7e service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:55.015263 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-5c35d39c-ff3c-4ff4-9c55-459927ecbb5d req-ded693ff-72db-410d-91f0-f4a0ea4d0f7e service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:55.016369 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-5c35d39c-ff3c-4ff4-9c55-459927ecbb5d req-ded693ff-72db-410d-91f0-f4a0ea4d0f7e service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:55.041513 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.compute.server_external_events [req-5c35d39c-ff3c-4ff4-9c55-459927ecbb5d req-ded693ff-72db-410d-91f0-f4a0ea4d0f7e service nova] Creating event network-changed:9b3bb00c-894f-4b64-935c-89a64519fd90 for instance a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 on hv-cinder-74014 Aug 08 02:27:55.057242 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [req-5c35d39c-ff3c-4ff4-9c55-459927ecbb5d req-ded693ff-72db-410d-91f0-f4a0ea4d0f7e service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.076082 Aug 08 02:27:55.077086 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 168/337] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:27:54 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 08 02:27:55.146987 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-06e57ac3-b323-4919-9fbd-4ce1e0dd1b1a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:55.154036 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-314a50b7-83e8-4045-b000-ef5d313f87be tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:55.154718 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-314a50b7-83e8-4045-b000-ef5d313f87be tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:55.164117 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-06e57ac3-b323-4919-9fbd-4ce1e0dd1b1a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61" status: 200 len: 1367 microversion: 2.1 time: 0.284583 Aug 08 02:27:55.164722 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 170/338] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:54 2022] GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 => generated 1367 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:55.183123 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-314a50b7-83e8-4045-b000-ef5d313f87be tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:55.185222 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-314a50b7-83e8-4045-b000-ef5d313f87be tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:55.185222 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-314a50b7-83e8-4045-b000-ef5d313f87be tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:55.185222 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-314a50b7-83e8-4045-b000-ef5d313f87be tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:55.185833 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-314a50b7-83e8-4045-b000-ef5d313f87be tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:55.187175 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-314a50b7-83e8-4045-b000-ef5d313f87be tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:55.367647 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-314a50b7-83e8-4045-b000-ef5d313f87be tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:55.381019 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-314a50b7-83e8-4045-b000-ef5d313f87be tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1712 microversion: 2.1 time: 0.231096 Aug 08 02:27:55.382604 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 169/339] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:55 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1712 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:55.382722 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-55dbeb42-0afb-421e-92cf-0645a14e99ae tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:55.383249 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-55dbeb42-0afb-421e-92cf-0645a14e99ae tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:55.390486 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-55dbeb42-0afb-421e-92cf-0645a14e99ae tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:55.391218 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-55dbeb42-0afb-421e-92cf-0645a14e99ae tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:55.391651 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-55dbeb42-0afb-421e-92cf-0645a14e99ae tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:55.392889 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-55dbeb42-0afb-421e-92cf-0645a14e99ae tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:55.393508 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-55dbeb42-0afb-421e-92cf-0645a14e99ae tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:55.394086 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-55dbeb42-0afb-421e-92cf-0645a14e99ae tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:55.584008 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-55dbeb42-0afb-421e-92cf-0645a14e99ae tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:55.596440 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-55dbeb42-0afb-421e-92cf-0645a14e99ae tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:55.598263 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-55dbeb42-0afb-421e-92cf-0645a14e99ae tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:55.599553 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-55dbeb42-0afb-421e-92cf-0645a14e99ae tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1746 microversion: 2.20 time: 0.223941 Aug 08 02:27:55.610494 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 171/340] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:27:55 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1746 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 08 02:27:56.187905 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-fb4e57ac-db6c-42e3-8ecf-35d8224cbfc4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:56.188534 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-fb4e57ac-db6c-42e3-8ecf-35d8224cbfc4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: dda0c0b0-ed8e-4d43-9b26-abe1bca05a61] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:56.195700 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-fb4e57ac-db6c-42e3-8ecf-35d8224cbfc4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:56.196385 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-fb4e57ac-db6c-42e3-8ecf-35d8224cbfc4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:56.196860 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-fb4e57ac-db6c-42e3-8ecf-35d8224cbfc4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:56.197556 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-fb4e57ac-db6c-42e3-8ecf-35d8224cbfc4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:56.198016 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-fb4e57ac-db6c-42e3-8ecf-35d8224cbfc4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:56.198437 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-fb4e57ac-db6c-42e3-8ecf-35d8224cbfc4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:56.365774 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-fb4e57ac-db6c-42e3-8ecf-35d8224cbfc4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:56.374453 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-fb4e57ac-db6c-42e3-8ecf-35d8224cbfc4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61" status: 200 len: 1367 microversion: 2.1 time: 0.190437 Aug 08 02:27:56.375407 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 170/341] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:56 2022] GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 => generated 1367 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:56.403655 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-264fda78-17a3-45bf-91a9-5c3cff7e66f1 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:56.404706 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-264fda78-17a3-45bf-91a9-5c3cff7e66f1 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:56.412685 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-264fda78-17a3-45bf-91a9-5c3cff7e66f1 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:56.413598 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-264fda78-17a3-45bf-91a9-5c3cff7e66f1 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:56.414767 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-264fda78-17a3-45bf-91a9-5c3cff7e66f1 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:56.415527 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-264fda78-17a3-45bf-91a9-5c3cff7e66f1 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:56.416146 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-264fda78-17a3-45bf-91a9-5c3cff7e66f1 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:56.419312 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-264fda78-17a3-45bf-91a9-5c3cff7e66f1 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:56.562944 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-264fda78-17a3-45bf-91a9-5c3cff7e66f1 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:56.571487 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-264fda78-17a3-45bf-91a9-5c3cff7e66f1 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1712 microversion: 2.1 time: 0.171956 Aug 08 02:27:56.572048 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 172/342] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:56 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1712 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:56.623329 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-42375f4b-718e-42ac-997a-020726c53f27 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:56.623971 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-42375f4b-718e-42ac-997a-020726c53f27 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:56.630631 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-42375f4b-718e-42ac-997a-020726c53f27 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:56.631113 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-42375f4b-718e-42ac-997a-020726c53f27 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:56.631548 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-42375f4b-718e-42ac-997a-020726c53f27 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:56.632238 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-42375f4b-718e-42ac-997a-020726c53f27 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:56.632710 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-42375f4b-718e-42ac-997a-020726c53f27 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:56.633135 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-42375f4b-718e-42ac-997a-020726c53f27 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:56.764761 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-42375f4b-718e-42ac-997a-020726c53f27 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:56.772904 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-42375f4b-718e-42ac-997a-020726c53f27 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:56.774420 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-42375f4b-718e-42ac-997a-020726c53f27 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:56.775563 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-42375f4b-718e-42ac-997a-020726c53f27 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1746 microversion: 2.20 time: 0.156195 Aug 08 02:27:56.776123 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 171/343] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:27:56 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1746 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:27:57.395784 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-3c51e785-6589-492f-a452-3aa6b9a556aa tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:57.396417 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-3c51e785-6589-492f-a452-3aa6b9a556aa tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: dda0c0b0-ed8e-4d43-9b26-abe1bca05a61] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:57.402346 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3c51e785-6589-492f-a452-3aa6b9a556aa tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:57.402823 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3c51e785-6589-492f-a452-3aa6b9a556aa tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:57.403247 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3c51e785-6589-492f-a452-3aa6b9a556aa tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:57.403921 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3c51e785-6589-492f-a452-3aa6b9a556aa tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:57.404503 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3c51e785-6589-492f-a452-3aa6b9a556aa tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:57.404899 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3c51e785-6589-492f-a452-3aa6b9a556aa tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:57.549684 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-3c51e785-6589-492f-a452-3aa6b9a556aa tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:57.557150 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-3c51e785-6589-492f-a452-3aa6b9a556aa tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61" status: 200 len: 1367 microversion: 2.1 time: 0.165268 Aug 08 02:27:57.557710 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 173/344] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:57 2022] GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 => generated 1367 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:27:57.592278 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-a9dcee12-d80b-4284-8a3b-a4099c23b5e3 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:57.594783 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-a9dcee12-d80b-4284-8a3b-a4099c23b5e3 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:57.601389 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a9dcee12-d80b-4284-8a3b-a4099c23b5e3 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:57.601871 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a9dcee12-d80b-4284-8a3b-a4099c23b5e3 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:57.602340 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a9dcee12-d80b-4284-8a3b-a4099c23b5e3 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:57.603038 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a9dcee12-d80b-4284-8a3b-a4099c23b5e3 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:57.603528 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a9dcee12-d80b-4284-8a3b-a4099c23b5e3 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:57.603951 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a9dcee12-d80b-4284-8a3b-a4099c23b5e3 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:57.730717 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-a9dcee12-d80b-4284-8a3b-a4099c23b5e3 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:57.739256 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-a9dcee12-d80b-4284-8a3b-a4099c23b5e3 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1712 microversion: 2.1 time: 0.150983 Aug 08 02:27:57.739825 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 172/345] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:57 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1712 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:57.799327 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-222de8bc-03d0-422a-8468-a26d46115d19 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:57.799885 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-222de8bc-03d0-422a-8468-a26d46115d19 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:57.806078 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-222de8bc-03d0-422a-8468-a26d46115d19 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:57.806563 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-222de8bc-03d0-422a-8468-a26d46115d19 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:57.807003 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-222de8bc-03d0-422a-8468-a26d46115d19 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:57.807692 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-222de8bc-03d0-422a-8468-a26d46115d19 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:57.808159 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-222de8bc-03d0-422a-8468-a26d46115d19 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:57.808584 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-222de8bc-03d0-422a-8468-a26d46115d19 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:57.937418 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-222de8bc-03d0-422a-8468-a26d46115d19 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:57.946025 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-222de8bc-03d0-422a-8468-a26d46115d19 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:57.947563 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-222de8bc-03d0-422a-8468-a26d46115d19 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:57.948829 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-222de8bc-03d0-422a-8468-a26d46115d19 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1746 microversion: 2.20 time: 0.154431 Aug 08 02:27:57.949386 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 174/346] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:27:57 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1746 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:27:58.578303 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-10edd232-c97d-421c-8246-42cb4f728779 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:58.578941 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-10edd232-c97d-421c-8246-42cb4f728779 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: dda0c0b0-ed8e-4d43-9b26-abe1bca05a61] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:58.585061 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-10edd232-c97d-421c-8246-42cb4f728779 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:58.585580 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-10edd232-c97d-421c-8246-42cb4f728779 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:58.586015 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-10edd232-c97d-421c-8246-42cb4f728779 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:58.586783 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-10edd232-c97d-421c-8246-42cb4f728779 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:58.587235 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-10edd232-c97d-421c-8246-42cb4f728779 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:58.587657 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-10edd232-c97d-421c-8246-42cb4f728779 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:58.715674 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-10edd232-c97d-421c-8246-42cb4f728779 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:58.719385 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:58.723625 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-10edd232-c97d-421c-8246-42cb4f728779 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61" status: 200 len: 1367 microversion: 2.1 time: 0.149311 Aug 08 02:27:58.724186 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 173/347] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:58 2022] GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 => generated 1367 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:58.759974 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-668ced5f-5678-4eed-9832-8710bc769c0a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:58.760618 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-668ced5f-5678-4eed-9832-8710bc769c0a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:58.767212 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-668ced5f-5678-4eed-9832-8710bc769c0a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:58.767719 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-668ced5f-5678-4eed-9832-8710bc769c0a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:58.768158 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-668ced5f-5678-4eed-9832-8710bc769c0a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:58.768889 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-668ced5f-5678-4eed-9832-8710bc769c0a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:58.769350 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-668ced5f-5678-4eed-9832-8710bc769c0a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:58.769780 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-668ced5f-5678-4eed-9832-8710bc769c0a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:58.902416 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-668ced5f-5678-4eed-9832-8710bc769c0a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:58.911068 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-668ced5f-5678-4eed-9832-8710bc769c0a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1712 microversion: 2.1 time: 0.154895 Aug 08 02:27:58.911649 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 175/348] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:58 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1712 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:27:58.972935 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-2a7607e6-dc5f-425d-bc63-61fac148e751 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:58.973735 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-2a7607e6-dc5f-425d-bc63-61fac148e751 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:58.980469 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2a7607e6-dc5f-425d-bc63-61fac148e751 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:58.981000 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2a7607e6-dc5f-425d-bc63-61fac148e751 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:58.981434 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2a7607e6-dc5f-425d-bc63-61fac148e751 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:58.982108 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2a7607e6-dc5f-425d-bc63-61fac148e751 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:58.982557 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2a7607e6-dc5f-425d-bc63-61fac148e751 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:58.983067 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2a7607e6-dc5f-425d-bc63-61fac148e751 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:59.115491 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-2a7607e6-dc5f-425d-bc63-61fac148e751 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:59.124093 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-2a7607e6-dc5f-425d-bc63-61fac148e751 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:59.125765 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-2a7607e6-dc5f-425d-bc63-61fac148e751 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:59.126969 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-2a7607e6-dc5f-425d-bc63-61fac148e751 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1746 microversion: 2.20 time: 0.159869 Aug 08 02:27:59.127526 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 174/349] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:27:58 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1746 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:27:59.747486 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-c9436c8e-232d-4bf8-ac92-6ab8aac43bb1 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:59.748067 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-c9436c8e-232d-4bf8-ac92-6ab8aac43bb1 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: dda0c0b0-ed8e-4d43-9b26-abe1bca05a61] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:59.755932 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c9436c8e-232d-4bf8-ac92-6ab8aac43bb1 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:59.756734 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c9436c8e-232d-4bf8-ac92-6ab8aac43bb1 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:59.756987 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c9436c8e-232d-4bf8-ac92-6ab8aac43bb1 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:59.757580 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c9436c8e-232d-4bf8-ac92-6ab8aac43bb1 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:59.758037 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c9436c8e-232d-4bf8-ac92-6ab8aac43bb1 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:59.758462 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c9436c8e-232d-4bf8-ac92-6ab8aac43bb1 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:59.897570 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-c9436c8e-232d-4bf8-ac92-6ab8aac43bb1 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:27:59.905695 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-c9436c8e-232d-4bf8-ac92-6ab8aac43bb1 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61" status: 200 len: 1367 microversion: 2.1 time: 0.162054 Aug 08 02:27:59.906443 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 176/350] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:59 2022] GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 => generated 1367 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:27:59.933559 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-f5640c4b-fb5d-4be6-92c5-d9fd8b42233a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:27:59.934165 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-f5640c4b-fb5d-4be6-92c5-d9fd8b42233a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:27:59.942073 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f5640c4b-fb5d-4be6-92c5-d9fd8b42233a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:59.942561 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f5640c4b-fb5d-4be6-92c5-d9fd8b42233a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:59.943153 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f5640c4b-fb5d-4be6-92c5-d9fd8b42233a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:27:59.943865 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f5640c4b-fb5d-4be6-92c5-d9fd8b42233a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:27:59.944325 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f5640c4b-fb5d-4be6-92c5-d9fd8b42233a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:27:59.944794 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f5640c4b-fb5d-4be6-92c5-d9fd8b42233a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:00.084372 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-f5640c4b-fb5d-4be6-92c5-d9fd8b42233a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:00.092427 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-f5640c4b-fb5d-4be6-92c5-d9fd8b42233a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1712 microversion: 2.1 time: 0.162762 Aug 08 02:28:00.093652 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 175/351] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:27:59 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1712 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:28:00.153142 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-8714ad08-6f05-4b65-9a1e-508c8c25a00d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:00.153796 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-8714ad08-6f05-4b65-9a1e-508c8c25a00d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:00.160058 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8714ad08-6f05-4b65-9a1e-508c8c25a00d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:00.160621 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8714ad08-6f05-4b65-9a1e-508c8c25a00d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:00.161122 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8714ad08-6f05-4b65-9a1e-508c8c25a00d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:00.161818 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8714ad08-6f05-4b65-9a1e-508c8c25a00d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:00.162368 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8714ad08-6f05-4b65-9a1e-508c8c25a00d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:00.162795 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8714ad08-6f05-4b65-9a1e-508c8c25a00d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:00.298641 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-8714ad08-6f05-4b65-9a1e-508c8c25a00d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:00.308169 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-8714ad08-6f05-4b65-9a1e-508c8c25a00d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:00.309784 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-8714ad08-6f05-4b65-9a1e-508c8c25a00d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:00.311042 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-8714ad08-6f05-4b65-9a1e-508c8c25a00d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1741 microversion: 2.20 time: 0.162166 Aug 08 02:28:00.312868 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 177/352] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:28:00 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1741 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:28:00.927764 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-6f60d54b-03cc-4af5-bf30-c0ca3468d7a0 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:00.928383 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-6f60d54b-03cc-4af5-bf30-c0ca3468d7a0 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: dda0c0b0-ed8e-4d43-9b26-abe1bca05a61] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:00.934947 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6f60d54b-03cc-4af5-bf30-c0ca3468d7a0 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:00.935614 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6f60d54b-03cc-4af5-bf30-c0ca3468d7a0 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:00.935928 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6f60d54b-03cc-4af5-bf30-c0ca3468d7a0 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:00.937424 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6f60d54b-03cc-4af5-bf30-c0ca3468d7a0 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:00.937916 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6f60d54b-03cc-4af5-bf30-c0ca3468d7a0 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:00.938341 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6f60d54b-03cc-4af5-bf30-c0ca3468d7a0 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:01.093735 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-6f60d54b-03cc-4af5-bf30-c0ca3468d7a0 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:01.101111 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-6f60d54b-03cc-4af5-bf30-c0ca3468d7a0 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61" status: 200 len: 1367 microversion: 2.1 time: 0.177301 Aug 08 02:28:01.101668 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 176/353] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:28:00 2022] GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 => generated 1367 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:28:01.118906 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-6b871566-634f-42c5-a8df-7e74cedcc8e9 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:01.119534 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-6b871566-634f-42c5-a8df-7e74cedcc8e9 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:01.127407 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6b871566-634f-42c5-a8df-7e74cedcc8e9 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:01.127907 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6b871566-634f-42c5-a8df-7e74cedcc8e9 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:01.128346 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6b871566-634f-42c5-a8df-7e74cedcc8e9 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:01.129073 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6b871566-634f-42c5-a8df-7e74cedcc8e9 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:01.129533 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6b871566-634f-42c5-a8df-7e74cedcc8e9 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:01.129979 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6b871566-634f-42c5-a8df-7e74cedcc8e9 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:01.335789 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-146c1486-1978-405f-abb9-f769d9ce47fc tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:01.336457 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-146c1486-1978-405f-abb9-f769d9ce47fc tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:01.349867 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-146c1486-1978-405f-abb9-f769d9ce47fc tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:01.353845 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-146c1486-1978-405f-abb9-f769d9ce47fc tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:01.354969 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-146c1486-1978-405f-abb9-f769d9ce47fc tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:01.359271 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-6b871566-634f-42c5-a8df-7e74cedcc8e9 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:01.360395 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-146c1486-1978-405f-abb9-f769d9ce47fc tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:01.362930 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-146c1486-1978-405f-abb9-f769d9ce47fc tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:01.366172 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-146c1486-1978-405f-abb9-f769d9ce47fc tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:01.370237 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-6b871566-634f-42c5-a8df-7e74cedcc8e9 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1716 microversion: 2.1 time: 0.256012 Aug 08 02:28:01.372523 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 178/354] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:28:01 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1716 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:28:01.608629 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-146c1486-1978-405f-abb9-f769d9ce47fc tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:01.618770 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-146c1486-1978-405f-abb9-f769d9ce47fc tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:01.620453 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-146c1486-1978-405f-abb9-f769d9ce47fc tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:01.621873 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-146c1486-1978-405f-abb9-f769d9ce47fc tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1741 microversion: 2.20 time: 0.290018 Aug 08 02:28:01.637261 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 177/355] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:28:01 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1741 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:28:01.696877 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [req-7e9ecbf5-94fd-435d-b69e-14bfd61804d4 req-d85edf17-baba-4b7d-bd42-cadbc2a43f82 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dda0c0b0-ed8e-4d43-9b26-abe1bca05a61", "name": "network-vif-plugged", "status": "completed", "tag": "c32805ad-aa79-4a4e-bcb8-fdaa7a625107"}]} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:28:01.708924 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-7e9ecbf5-94fd-435d-b69e-14bfd61804d4 req-d85edf17-baba-4b7d-bd42-cadbc2a43f82 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:01.709689 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-7e9ecbf5-94fd-435d-b69e-14bfd61804d4 req-d85edf17-baba-4b7d-bd42-cadbc2a43f82 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:01.710272 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-7e9ecbf5-94fd-435d-b69e-14bfd61804d4 req-d85edf17-baba-4b7d-bd42-cadbc2a43f82 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:01.741446 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.compute.server_external_events [req-7e9ecbf5-94fd-435d-b69e-14bfd61804d4 req-d85edf17-baba-4b7d-bd42-cadbc2a43f82 service nova] Creating event network-vif-plugged:c32805ad-aa79-4a4e-bcb8-fdaa7a625107 for instance dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 on hv-cinder-74014 Aug 08 02:28:01.741446 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [req-7e9ecbf5-94fd-435d-b69e-14bfd61804d4 req-d85edf17-baba-4b7d-bd42-cadbc2a43f82 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.048613 Aug 08 02:28:01.741819 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 179/356] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:28:01 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:28:02.122639 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-fa49e2eb-7e19-4301-8f2a-ef5a8127c215 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:02.124551 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-fa49e2eb-7e19-4301-8f2a-ef5a8127c215 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: dda0c0b0-ed8e-4d43-9b26-abe1bca05a61] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:02.143999 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-fa49e2eb-7e19-4301-8f2a-ef5a8127c215 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:02.145630 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-fa49e2eb-7e19-4301-8f2a-ef5a8127c215 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:02.146511 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-fa49e2eb-7e19-4301-8f2a-ef5a8127c215 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:02.150479 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-fa49e2eb-7e19-4301-8f2a-ef5a8127c215 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:02.154295 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-fa49e2eb-7e19-4301-8f2a-ef5a8127c215 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:02.156119 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-fa49e2eb-7e19-4301-8f2a-ef5a8127c215 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:02.421577 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-fa49e2eb-7e19-4301-8f2a-ef5a8127c215 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:02.429870 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-fa49e2eb-7e19-4301-8f2a-ef5a8127c215 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61" status: 200 len: 1367 microversion: 2.1 time: 0.311700 Aug 08 02:28:02.430450 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 178/357] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:28:02 2022] GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 => generated 1367 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:28:02.651879 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-b07fffdb-3c0e-4c19-8a10-66f23be08d12 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:02.653425 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-b07fffdb-3c0e-4c19-8a10-66f23be08d12 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:02.674523 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b07fffdb-3c0e-4c19-8a10-66f23be08d12 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:02.873782 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b07fffdb-3c0e-4c19-8a10-66f23be08d12 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:02.873782 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b07fffdb-3c0e-4c19-8a10-66f23be08d12 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:02.873782 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b07fffdb-3c0e-4c19-8a10-66f23be08d12 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:02.873782 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b07fffdb-3c0e-4c19-8a10-66f23be08d12 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:02.874300 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b07fffdb-3c0e-4c19-8a10-66f23be08d12 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:02.874300 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-b07fffdb-3c0e-4c19-8a10-66f23be08d12 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:02.874584 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-b07fffdb-3c0e-4c19-8a10-66f23be08d12 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:02.874584 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-b07fffdb-3c0e-4c19-8a10-66f23be08d12 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:02.875407 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-b07fffdb-3c0e-4c19-8a10-66f23be08d12 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1741 microversion: 2.20 time: 0.228431 Aug 08 02:28:02.876302 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 180/358] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:28:02 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1741 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:28:03.451327 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-c54b9152-dc1b-4a32-b62e-78c973e99e96 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:03.452107 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-c54b9152-dc1b-4a32-b62e-78c973e99e96 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: dda0c0b0-ed8e-4d43-9b26-abe1bca05a61] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:03.463408 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c54b9152-dc1b-4a32-b62e-78c973e99e96 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:03.468394 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c54b9152-dc1b-4a32-b62e-78c973e99e96 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:03.468394 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c54b9152-dc1b-4a32-b62e-78c973e99e96 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:03.468394 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c54b9152-dc1b-4a32-b62e-78c973e99e96 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:03.468394 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c54b9152-dc1b-4a32-b62e-78c973e99e96 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:03.468935 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c54b9152-dc1b-4a32-b62e-78c973e99e96 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:03.645967 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-c54b9152-dc1b-4a32-b62e-78c973e99e96 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:03.663381 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-c54b9152-dc1b-4a32-b62e-78c973e99e96 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61" status: 200 len: 1367 microversion: 2.1 time: 0.214597 Aug 08 02:28:03.663381 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 179/359] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:28:03 2022] GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 => generated 1367 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:28:03.756892 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [req-9ddfbd2d-b8f0-460f-a854-d3265b37e7fb req-0fd7bfb6-8dfd-451c-8b17-9c01dd274cf0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f51a5900-5abc-4188-bbc4-33b0f0f7d091", "name": "network-vif-plugged", "status": "completed", "tag": "8d2a6aac-ff75-417c-b0b9-cbf799f88d22"}]} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:28:03.768216 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-9ddfbd2d-b8f0-460f-a854-d3265b37e7fb req-0fd7bfb6-8dfd-451c-8b17-9c01dd274cf0 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:03.769454 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-9ddfbd2d-b8f0-460f-a854-d3265b37e7fb req-0fd7bfb6-8dfd-451c-8b17-9c01dd274cf0 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:03.770273 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-9ddfbd2d-b8f0-460f-a854-d3265b37e7fb req-0fd7bfb6-8dfd-451c-8b17-9c01dd274cf0 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:03.784954 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.compute.server_external_events [req-9ddfbd2d-b8f0-460f-a854-d3265b37e7fb req-0fd7bfb6-8dfd-451c-8b17-9c01dd274cf0 service nova] Creating event network-vif-plugged:8d2a6aac-ff75-417c-b0b9-cbf799f88d22 for instance f51a5900-5abc-4188-bbc4-33b0f0f7d091 on hv-cinder-74014 Aug 08 02:28:03.790448 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [req-9ddfbd2d-b8f0-460f-a854-d3265b37e7fb req-0fd7bfb6-8dfd-451c-8b17-9c01dd274cf0 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037561 Aug 08 02:28:03.791124 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 181/360] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:28:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:28:03.908616 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-8a851e36-336f-4b18-bb94-5396dd105288 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:03.910204 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-8a851e36-336f-4b18-bb94-5396dd105288 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:03.920104 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8a851e36-336f-4b18-bb94-5396dd105288 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:03.927248 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8a851e36-336f-4b18-bb94-5396dd105288 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:03.927248 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8a851e36-336f-4b18-bb94-5396dd105288 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:03.927248 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8a851e36-336f-4b18-bb94-5396dd105288 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:03.927248 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8a851e36-336f-4b18-bb94-5396dd105288 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:03.927602 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8a851e36-336f-4b18-bb94-5396dd105288 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:04.126277 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-8a851e36-336f-4b18-bb94-5396dd105288 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:04.142238 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-8a851e36-336f-4b18-bb94-5396dd105288 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:04.144311 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-8a851e36-336f-4b18-bb94-5396dd105288 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:04.145830 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-8a851e36-336f-4b18-bb94-5396dd105288 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1741 microversion: 2.20 time: 0.250917 Aug 08 02:28:04.146622 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 180/361] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:28:03 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1741 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:28:04.686769 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-d61d0e64-3a9a-42d2-a4c0-1676684a4398 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:04.688510 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-d61d0e64-3a9a-42d2-a4c0-1676684a4398 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: dda0c0b0-ed8e-4d43-9b26-abe1bca05a61] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:04.702021 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d61d0e64-3a9a-42d2-a4c0-1676684a4398 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:04.703198 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d61d0e64-3a9a-42d2-a4c0-1676684a4398 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:04.703737 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d61d0e64-3a9a-42d2-a4c0-1676684a4398 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:04.704502 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d61d0e64-3a9a-42d2-a4c0-1676684a4398 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:04.705358 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d61d0e64-3a9a-42d2-a4c0-1676684a4398 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:04.705891 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d61d0e64-3a9a-42d2-a4c0-1676684a4398 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:04.868380 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-d61d0e64-3a9a-42d2-a4c0-1676684a4398 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:04.877796 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-d61d0e64-3a9a-42d2-a4c0-1676684a4398 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61" status: 200 len: 1558 microversion: 2.1 time: 0.197593 Aug 08 02:28:04.889114 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 182/362] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:28:04 2022] GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 => generated 1558 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:28:04.899551 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-3aa64da2-2a70-43c3-99a2-b7982912e04c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:04.900096 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-3aa64da2-2a70-43c3-99a2-b7982912e04c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: dda0c0b0-ed8e-4d43-9b26-abe1bca05a61] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:04.906777 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3aa64da2-2a70-43c3-99a2-b7982912e04c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:04.907457 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3aa64da2-2a70-43c3-99a2-b7982912e04c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:04.907888 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3aa64da2-2a70-43c3-99a2-b7982912e04c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:04.908571 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3aa64da2-2a70-43c3-99a2-b7982912e04c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:04.909048 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3aa64da2-2a70-43c3-99a2-b7982912e04c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:04.910128 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3aa64da2-2a70-43c3-99a2-b7982912e04c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:05.170381 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-8a5579a2-f2d6-49ec-9573-a31d2a21cd15 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:05.171827 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-8a5579a2-f2d6-49ec-9573-a31d2a21cd15 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:05.178334 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8a5579a2-f2d6-49ec-9573-a31d2a21cd15 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:05.179138 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8a5579a2-f2d6-49ec-9573-a31d2a21cd15 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:05.179791 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8a5579a2-f2d6-49ec-9573-a31d2a21cd15 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:05.180505 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8a5579a2-f2d6-49ec-9573-a31d2a21cd15 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:05.181188 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8a5579a2-f2d6-49ec-9573-a31d2a21cd15 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:05.181623 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8a5579a2-f2d6-49ec-9573-a31d2a21cd15 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:05.258512 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-3aa64da2-2a70-43c3-99a2-b7982912e04c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.361004 Aug 08 02:28:05.259593 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 181/363] 10.222.0.27 () {58 vars in 1275 bytes} [Mon Aug 8 02:28:04 2022] GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61/os-interface => generated 285 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:28:05.420249 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-8a5579a2-f2d6-49ec-9573-a31d2a21cd15 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:05.443013 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-8a5579a2-f2d6-49ec-9573-a31d2a21cd15 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:05.445024 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-8a5579a2-f2d6-49ec-9573-a31d2a21cd15 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:05.446564 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-8a5579a2-f2d6-49ec-9573-a31d2a21cd15 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1741 microversion: 2.20 time: 0.280244 Aug 08 02:28:05.447607 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 183/364] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:28:05 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1741 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 08 02:28:06.476832 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-857d2b33-d990-490a-b0ac-2af55ceeb3c1 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:06.477370 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-857d2b33-d990-490a-b0ac-2af55ceeb3c1 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:06.515889 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-857d2b33-d990-490a-b0ac-2af55ceeb3c1 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:06.520342 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-857d2b33-d990-490a-b0ac-2af55ceeb3c1 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:06.520977 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-857d2b33-d990-490a-b0ac-2af55ceeb3c1 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:06.521725 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-857d2b33-d990-490a-b0ac-2af55ceeb3c1 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:06.522354 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-857d2b33-d990-490a-b0ac-2af55ceeb3c1 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:06.522795 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-857d2b33-d990-490a-b0ac-2af55ceeb3c1 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:06.829830 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [req-511dd41d-f11d-41a0-8de6-18b15fc7560e req-c50986aa-76dd-41cb-9d4c-26184616d0e4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "dda0c0b0-ed8e-4d43-9b26-abe1bca05a61", "tag": "c32805ad-aa79-4a4e-bcb8-fdaa7a625107"}]} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:28:06.851771 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-511dd41d-f11d-41a0-8de6-18b15fc7560e req-c50986aa-76dd-41cb-9d4c-26184616d0e4 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:06.854959 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-511dd41d-f11d-41a0-8de6-18b15fc7560e req-c50986aa-76dd-41cb-9d4c-26184616d0e4 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:06.855763 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-511dd41d-f11d-41a0-8de6-18b15fc7560e req-c50986aa-76dd-41cb-9d4c-26184616d0e4 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:06.875649 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell1:SELECT=100 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:06.908697 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.compute.server_external_events [req-511dd41d-f11d-41a0-8de6-18b15fc7560e req-c50986aa-76dd-41cb-9d4c-26184616d0e4 service nova] Creating event network-changed:c32805ad-aa79-4a4e-bcb8-fdaa7a625107 for instance dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 on hv-cinder-74014 Aug 08 02:28:06.928170 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [req-511dd41d-f11d-41a0-8de6-18b15fc7560e req-c50986aa-76dd-41cb-9d4c-26184616d0e4 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.093671 Aug 08 02:28:06.941903 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 184/365] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:28:06 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 08 02:28:06.956455 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-857d2b33-d990-490a-b0ac-2af55ceeb3c1 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:06.966949 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-857d2b33-d990-490a-b0ac-2af55ceeb3c1 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:06.977335 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-857d2b33-d990-490a-b0ac-2af55ceeb3c1 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:06.982244 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-857d2b33-d990-490a-b0ac-2af55ceeb3c1 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1741 microversion: 2.20 time: 0.507473 Aug 08 02:28:06.982244 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 182/366] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:28:06 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1741 bytes in 509 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 08 02:28:08.008299 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-a6e182ec-540f-4be0-ac6a-3f0c19d54704 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:08.009087 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-a6e182ec-540f-4be0-ac6a-3f0c19d54704 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:08.023637 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a6e182ec-540f-4be0-ac6a-3f0c19d54704 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:08.024165 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a6e182ec-540f-4be0-ac6a-3f0c19d54704 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:08.031540 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a6e182ec-540f-4be0-ac6a-3f0c19d54704 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:08.032394 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a6e182ec-540f-4be0-ac6a-3f0c19d54704 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:08.045648 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a6e182ec-540f-4be0-ac6a-3f0c19d54704 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:08.045648 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a6e182ec-540f-4be0-ac6a-3f0c19d54704 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:08.362623 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-a6e182ec-540f-4be0-ac6a-3f0c19d54704 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:08.384642 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-a6e182ec-540f-4be0-ac6a-3f0c19d54704 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:08.387871 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-a6e182ec-540f-4be0-ac6a-3f0c19d54704 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:08.392278 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-a6e182ec-540f-4be0-ac6a-3f0c19d54704 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1741 microversion: 2.20 time: 0.386075 Aug 08 02:28:08.409863 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 185/367] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:28:08 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1741 bytes in 406 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:28:09.416734 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-1c4233f4-477b-419a-8ab1-c11fd147c4cd tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:09.417493 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-1c4233f4-477b-419a-8ab1-c11fd147c4cd tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:09.424373 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1c4233f4-477b-419a-8ab1-c11fd147c4cd tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:09.424899 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1c4233f4-477b-419a-8ab1-c11fd147c4cd tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:09.425332 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1c4233f4-477b-419a-8ab1-c11fd147c4cd tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:09.426012 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1c4233f4-477b-419a-8ab1-c11fd147c4cd tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:09.426462 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1c4233f4-477b-419a-8ab1-c11fd147c4cd tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:09.426881 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1c4233f4-477b-419a-8ab1-c11fd147c4cd tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:09.557475 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-1c4233f4-477b-419a-8ab1-c11fd147c4cd tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:09.566157 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-1c4233f4-477b-419a-8ab1-c11fd147c4cd tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:09.567685 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-1c4233f4-477b-419a-8ab1-c11fd147c4cd tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:09.568881 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-1c4233f4-477b-419a-8ab1-c11fd147c4cd tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1741 microversion: 2.20 time: 0.156206 Aug 08 02:28:09.569445 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 183/368] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:28:09 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1741 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:28:10.600537 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-8cf41434-1d91-41f7-a8ac-41268449130b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:10.600988 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-8cf41434-1d91-41f7-a8ac-41268449130b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:10.607730 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8cf41434-1d91-41f7-a8ac-41268449130b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:10.608610 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8cf41434-1d91-41f7-a8ac-41268449130b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:10.609423 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8cf41434-1d91-41f7-a8ac-41268449130b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:10.610371 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8cf41434-1d91-41f7-a8ac-41268449130b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:10.611089 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8cf41434-1d91-41f7-a8ac-41268449130b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:10.611849 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8cf41434-1d91-41f7-a8ac-41268449130b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:10.775756 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-8cf41434-1d91-41f7-a8ac-41268449130b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:10.787938 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-8cf41434-1d91-41f7-a8ac-41268449130b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:10.795654 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-8cf41434-1d91-41f7-a8ac-41268449130b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:10.795654 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-8cf41434-1d91-41f7-a8ac-41268449130b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1741 microversion: 2.20 time: 0.201886 Aug 08 02:28:10.795654 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 186/369] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:28:10 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1741 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:28:11.818555 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-d6540c40-dd04-4cdf-a4cf-c38952f0582b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:11.819829 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-d6540c40-dd04-4cdf-a4cf-c38952f0582b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:11.834109 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d6540c40-dd04-4cdf-a4cf-c38952f0582b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:11.834109 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d6540c40-dd04-4cdf-a4cf-c38952f0582b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:11.834109 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d6540c40-dd04-4cdf-a4cf-c38952f0582b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:11.834977 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d6540c40-dd04-4cdf-a4cf-c38952f0582b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:11.839302 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d6540c40-dd04-4cdf-a4cf-c38952f0582b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:11.840290 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d6540c40-dd04-4cdf-a4cf-c38952f0582b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:12.087297 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-d6540c40-dd04-4cdf-a4cf-c38952f0582b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:12.102160 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-d6540c40-dd04-4cdf-a4cf-c38952f0582b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:12.113772 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-d6540c40-dd04-4cdf-a4cf-c38952f0582b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:12.118000 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-d6540c40-dd04-4cdf-a4cf-c38952f0582b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1741 microversion: 2.20 time: 0.303948 Aug 08 02:28:12.120927 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 184/370] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:28:11 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1741 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 08 02:28:13.145322 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-db805504-6795-47b1-8d63-b2bbbd5714a9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:13.154064 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-db805504-6795-47b1-8d63-b2bbbd5714a9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:13.154064 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-db805504-6795-47b1-8d63-b2bbbd5714a9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:13.154064 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-db805504-6795-47b1-8d63-b2bbbd5714a9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:13.154064 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-db805504-6795-47b1-8d63-b2bbbd5714a9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:13.154667 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-db805504-6795-47b1-8d63-b2bbbd5714a9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:13.154667 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-db805504-6795-47b1-8d63-b2bbbd5714a9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:13.154961 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-db805504-6795-47b1-8d63-b2bbbd5714a9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:13.347041 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-db805504-6795-47b1-8d63-b2bbbd5714a9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:13.355621 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-db805504-6795-47b1-8d63-b2bbbd5714a9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:13.357165 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-db805504-6795-47b1-8d63-b2bbbd5714a9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:13.358391 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-db805504-6795-47b1-8d63-b2bbbd5714a9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1741 microversion: 2.20 time: 0.218395 Aug 08 02:28:13.359221 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 187/371] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:28:13 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1741 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:28:14.383707 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-a1ba3521-7378-494b-9657-be96b9ae2845 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:14.385137 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-a1ba3521-7378-494b-9657-be96b9ae2845 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:14.391230 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_api:SELECT=74 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:14.399959 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a1ba3521-7378-494b-9657-be96b9ae2845 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:14.400706 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a1ba3521-7378-494b-9657-be96b9ae2845 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:14.401227 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a1ba3521-7378-494b-9657-be96b9ae2845 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:14.402467 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a1ba3521-7378-494b-9657-be96b9ae2845 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:14.402936 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a1ba3521-7378-494b-9657-be96b9ae2845 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:14.403372 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a1ba3521-7378-494b-9657-be96b9ae2845 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:14.623852 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-a1ba3521-7378-494b-9657-be96b9ae2845 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:14.633404 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-a1ba3521-7378-494b-9657-be96b9ae2845 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:14.635655 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-a1ba3521-7378-494b-9657-be96b9ae2845 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:14.637137 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-a1ba3521-7378-494b-9657-be96b9ae2845 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1741 microversion: 2.20 time: 0.259121 Aug 08 02:28:14.640705 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 185/372] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:28:14 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1741 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:28:15.661172 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-8daec8c4-2c73-4d06-ac23-9d1f2bbb6b75 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:15.661943 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-8daec8c4-2c73-4d06-ac23-9d1f2bbb6b75 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:15.664523 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_api:SELECT=76 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:15.669420 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8daec8c4-2c73-4d06-ac23-9d1f2bbb6b75 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:15.669899 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8daec8c4-2c73-4d06-ac23-9d1f2bbb6b75 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:15.670328 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8daec8c4-2c73-4d06-ac23-9d1f2bbb6b75 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:15.672318 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8daec8c4-2c73-4d06-ac23-9d1f2bbb6b75 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:15.672905 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8daec8c4-2c73-4d06-ac23-9d1f2bbb6b75 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:15.673474 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8daec8c4-2c73-4d06-ac23-9d1f2bbb6b75 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:15.826320 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-8daec8c4-2c73-4d06-ac23-9d1f2bbb6b75 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:15.839537 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-8daec8c4-2c73-4d06-ac23-9d1f2bbb6b75 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:15.839537 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-8daec8c4-2c73-4d06-ac23-9d1f2bbb6b75 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:15.839537 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-8daec8c4-2c73-4d06-ac23-9d1f2bbb6b75 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1741 microversion: 2.20 time: 0.181331 Aug 08 02:28:15.839866 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 188/373] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:28:15 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1741 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:28:16.869505 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-638cff46-b3b3-4023-a17d-865cbae8e831 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:16.869505 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-638cff46-b3b3-4023-a17d-865cbae8e831 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:16.870264 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-638cff46-b3b3-4023-a17d-865cbae8e831 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:16.870375 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-638cff46-b3b3-4023-a17d-865cbae8e831 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:16.870752 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-638cff46-b3b3-4023-a17d-865cbae8e831 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:16.871454 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-638cff46-b3b3-4023-a17d-865cbae8e831 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:16.871909 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-638cff46-b3b3-4023-a17d-865cbae8e831 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:16.872339 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-638cff46-b3b3-4023-a17d-865cbae8e831 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:17.015602 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-638cff46-b3b3-4023-a17d-865cbae8e831 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:17.024032 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-638cff46-b3b3-4023-a17d-865cbae8e831 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:17.025587 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-638cff46-b3b3-4023-a17d-865cbae8e831 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:17.026738 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-638cff46-b3b3-4023-a17d-865cbae8e831 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1723 microversion: 2.20 time: 0.168531 Aug 08 02:28:17.027291 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 186/374] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:28:16 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1723 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:28:18.052414 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-5699cede-d205-429a-8b4c-6c911a2f7375 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:18.052978 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-5699cede-d205-429a-8b4c-6c911a2f7375 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:18.059343 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5699cede-d205-429a-8b4c-6c911a2f7375 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:18.059826 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5699cede-d205-429a-8b4c-6c911a2f7375 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:18.060266 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5699cede-d205-429a-8b4c-6c911a2f7375 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:18.060978 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5699cede-d205-429a-8b4c-6c911a2f7375 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:18.061434 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5699cede-d205-429a-8b4c-6c911a2f7375 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:18.061858 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5699cede-d205-429a-8b4c-6c911a2f7375 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:18.193769 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-5699cede-d205-429a-8b4c-6c911a2f7375 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:18.203089 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-5699cede-d205-429a-8b4c-6c911a2f7375 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:18.204623 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-5699cede-d205-429a-8b4c-6c911a2f7375 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:18.205804 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-5699cede-d205-429a-8b4c-6c911a2f7375 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1723 microversion: 2.20 time: 0.157407 Aug 08 02:28:18.206365 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 189/375] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:28:18 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1723 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 08 02:28:19.230446 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-3cfa66ae-e125-4b6c-8808-111821e4b00e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:19.231895 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-3cfa66ae-e125-4b6c-8808-111821e4b00e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:19.239486 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3cfa66ae-e125-4b6c-8808-111821e4b00e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:19.241248 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3cfa66ae-e125-4b6c-8808-111821e4b00e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:19.245490 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3cfa66ae-e125-4b6c-8808-111821e4b00e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:19.245490 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3cfa66ae-e125-4b6c-8808-111821e4b00e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:19.245490 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3cfa66ae-e125-4b6c-8808-111821e4b00e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:19.245490 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3cfa66ae-e125-4b6c-8808-111821e4b00e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:19.440889 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-3cfa66ae-e125-4b6c-8808-111821e4b00e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:19.450171 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-3cfa66ae-e125-4b6c-8808-111821e4b00e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:19.457543 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-3cfa66ae-e125-4b6c-8808-111821e4b00e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:19.457543 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-3cfa66ae-e125-4b6c-8808-111821e4b00e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1726 microversion: 2.20 time: 0.226822 Aug 08 02:28:19.457543 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 187/376] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:28:19 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1726 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:28:19.472046 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [req-a32cf7ff-6321-4f23-a1c3-91c78e5586f8 req-7b64a23a-86a6-4359-8e84-05ac775aa32e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f51a5900-5abc-4188-bbc4-33b0f0f7d091", "name": "network-vif-unplugged", "status": "completed", "tag": "8d2a6aac-ff75-417c-b0b9-cbf799f88d22"}]} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:28:19.480780 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-a9d9d641-027f-43c7-b503-2be33f6e12a4 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "5132860c-23b8-4d38-9a6c-3a0487900b41"}} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:28:19.484414 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-a9d9d641-027f-43c7-b503-2be33f6e12a4 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:19.485013 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-a32cf7ff-6321-4f23-a1c3-91c78e5586f8 req-7b64a23a-86a6-4359-8e84-05ac775aa32e service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:19.485332 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-a32cf7ff-6321-4f23-a1c3-91c78e5586f8 req-7b64a23a-86a6-4359-8e84-05ac775aa32e service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:19.485684 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-a32cf7ff-6321-4f23-a1c3-91c78e5586f8 req-7b64a23a-86a6-4359-8e84-05ac775aa32e service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:19.491369 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a9d9d641-027f-43c7-b503-2be33f6e12a4 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:19.492005 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a9d9d641-027f-43c7-b503-2be33f6e12a4 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:19.493053 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a9d9d641-027f-43c7-b503-2be33f6e12a4 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:19.493355 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a9d9d641-027f-43c7-b503-2be33f6e12a4 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:19.493883 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a9d9d641-027f-43c7-b503-2be33f6e12a4 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:19.494320 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a9d9d641-027f-43c7-b503-2be33f6e12a4 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:19.498564 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.compute.server_external_events [req-a32cf7ff-6321-4f23-a1c3-91c78e5586f8 req-7b64a23a-86a6-4359-8e84-05ac775aa32e service nova] Creating event network-vif-unplugged:8d2a6aac-ff75-417c-b0b9-cbf799f88d22 for instance f51a5900-5abc-4188-bbc4-33b0f0f7d091 on hv-cinder-74014 Aug 08 02:28:19.504540 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [req-a32cf7ff-6321-4f23-a1c3-91c78e5586f8 req-7b64a23a-86a6-4359-8e84-05ac775aa32e service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.036692 Aug 08 02:28:19.505494 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 190/377] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:28:19 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 08 02:28:20.822824 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell1:SELECT=95,nova_cell1:INSERT=3,nova_cell1:UPDATE=2 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:20.904783 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-a9d9d641-027f-43c7-b503-2be33f6e12a4 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "POST /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 1.428677 Aug 08 02:28:20.906040 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 188/378] 10.222.0.27 () {62 vars in 1364 bytes} [Mon Aug 8 02:28:19 2022] POST /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091/os-volume_attachments => generated 188 bytes in 1431 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 08 02:28:21.035664 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-142a33ec-b162-4731-abf4-d4fe8b5ab129 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:28:21.036370 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-142a33ec-b162-4731-abf4-d4fe8b5ab129 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:21.042542 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-142a33ec-b162-4731-abf4-d4fe8b5ab129 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:21.043025 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-142a33ec-b162-4731-abf4-d4fe8b5ab129 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:21.043492 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-142a33ec-b162-4731-abf4-d4fe8b5ab129 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:21.044182 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-142a33ec-b162-4731-abf4-d4fe8b5ab129 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:21.044632 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-142a33ec-b162-4731-abf4-d4fe8b5ab129 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:21.045087 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-142a33ec-b162-4731-abf4-d4fe8b5ab129 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:21.091780 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-142a33ec-b162-4731-abf4-d4fe8b5ab129 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=118351) unshelve /opt/stack/nova/nova/compute/api.py:4531}} Aug 08 02:28:21.626575 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-142a33ec-b162-4731-abf4-d4fe8b5ab129 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "POST /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091/action" status: 202 len: 0 microversion: 2.20 time: 0.596259 Aug 08 02:28:21.627270 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 191/379] 10.222.0.27 () {62 vars in 1319 bytes} [Mon Aug 8 02:28:21 2022] POST /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091/action => generated 0 bytes in 598 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Aug 08 02:28:21.635499 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-4f24bdc8-6a36-4c9b-9687-ab4ff5ecd315 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:21.636116 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-4f24bdc8-6a36-4c9b-9687-ab4ff5ecd315 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:21.643420 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4f24bdc8-6a36-4c9b-9687-ab4ff5ecd315 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:21.643902 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4f24bdc8-6a36-4c9b-9687-ab4ff5ecd315 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:21.644337 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4f24bdc8-6a36-4c9b-9687-ab4ff5ecd315 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:21.645056 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4f24bdc8-6a36-4c9b-9687-ab4ff5ecd315 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:21.645508 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4f24bdc8-6a36-4c9b-9687-ab4ff5ecd315 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:21.645937 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4f24bdc8-6a36-4c9b-9687-ab4ff5ecd315 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:21.870760 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-4f24bdc8-6a36-4c9b-9687-ab4ff5ecd315 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:21.880533 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-4f24bdc8-6a36-4c9b-9687-ab4ff5ecd315 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:21.882201 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-4f24bdc8-6a36-4c9b-9687-ab4ff5ecd315 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:21.883348 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-4f24bdc8-6a36-4c9b-9687-ab4ff5ecd315 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1752 microversion: 2.20 time: 0.251814 Aug 08 02:28:21.883968 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 189/380] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:28:21 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1752 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:28:22.909039 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-5867c248-f149-4513-a7b4-8a76060877c9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:22.910533 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-5867c248-f149-4513-a7b4-8a76060877c9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:22.922086 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5867c248-f149-4513-a7b4-8a76060877c9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:22.922921 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5867c248-f149-4513-a7b4-8a76060877c9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:22.923595 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5867c248-f149-4513-a7b4-8a76060877c9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:22.925292 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5867c248-f149-4513-a7b4-8a76060877c9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:22.935307 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5867c248-f149-4513-a7b4-8a76060877c9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:22.936013 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5867c248-f149-4513-a7b4-8a76060877c9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:23.247062 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-5867c248-f149-4513-a7b4-8a76060877c9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:23.256472 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-5867c248-f149-4513-a7b4-8a76060877c9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:23.258025 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-5867c248-f149-4513-a7b4-8a76060877c9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:23.259519 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-5867c248-f149-4513-a7b4-8a76060877c9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1752 microversion: 2.20 time: 0.355814 Aug 08 02:28:23.259804 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 192/381] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:28:22 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1752 bytes in 357 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:28:24.286633 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-40ba1f6d-fbb6-4cfc-9fbf-604427fa1a7f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:24.287821 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-40ba1f6d-fbb6-4cfc-9fbf-604427fa1a7f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:24.313630 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-40ba1f6d-fbb6-4cfc-9fbf-604427fa1a7f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:24.314491 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-40ba1f6d-fbb6-4cfc-9fbf-604427fa1a7f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:24.315048 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-40ba1f6d-fbb6-4cfc-9fbf-604427fa1a7f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:24.316448 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-40ba1f6d-fbb6-4cfc-9fbf-604427fa1a7f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:24.317082 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-40ba1f6d-fbb6-4cfc-9fbf-604427fa1a7f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:24.318075 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-40ba1f6d-fbb6-4cfc-9fbf-604427fa1a7f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:24.458931 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-40ba1f6d-fbb6-4cfc-9fbf-604427fa1a7f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:24.467499 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-40ba1f6d-fbb6-4cfc-9fbf-604427fa1a7f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:24.469763 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-40ba1f6d-fbb6-4cfc-9fbf-604427fa1a7f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:24.470907 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-40ba1f6d-fbb6-4cfc-9fbf-604427fa1a7f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1754 microversion: 2.20 time: 0.189599 Aug 08 02:28:24.471464 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 190/382] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:28:24 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1754 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:28:25.494586 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-254bd19c-7972-453f-a949-51447c3ca9e5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:25.495590 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-254bd19c-7972-453f-a949-51447c3ca9e5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:25.502369 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-7db1bc16-e859-4b4c-a0e8-6b9653dc824b tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-1155722721", "imageRef": "30833f47-5987-41ca-b5ee-ad5ecafbb265", "flavorRef": "452", "networks": [{"uuid": "80d55d70-3b18-4f03-aa9c-500250bb4b23"}]}} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:28:25.503497 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-254bd19c-7972-453f-a949-51447c3ca9e5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:25.504180 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-254bd19c-7972-453f-a949-51447c3ca9e5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:25.504615 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-254bd19c-7972-453f-a949-51447c3ca9e5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:25.505514 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-254bd19c-7972-453f-a949-51447c3ca9e5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:25.505973 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-254bd19c-7972-453f-a949-51447c3ca9e5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:25.506398 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-254bd19c-7972-453f-a949-51447c3ca9e5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:25.579702 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.network.neutron [None req-7db1bc16-e859-4b4c-a0e8-6b9653dc824b tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] validate_networks() for [('80d55d70-3b18-4f03-aa9c-500250bb4b23', None, None, None, None, None)] {{(pid=118352) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 08 02:28:25.719280 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-254bd19c-7972-453f-a949-51447c3ca9e5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:25.732943 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-254bd19c-7972-453f-a949-51447c3ca9e5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:25.740860 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-254bd19c-7972-453f-a949-51447c3ca9e5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:25.740860 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-254bd19c-7972-453f-a949-51447c3ca9e5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1754 microversion: 2.20 time: 0.246062 Aug 08 02:28:25.740860 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 193/383] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:28:25 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1754 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:28:26.004677 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-7db1bc16-e859-4b4c-a0e8-6b9653dc824b tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Flavor limits 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 08 02:28:26.005269 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-7db1bc16-e859-4b4c-a0e8-6b9653dc824b tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Image limits 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 08 02:28:26.005742 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-7db1bc16-e859-4b4c-a0e8-6b9653dc824b tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Flavor pref 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 08 02:28:26.006176 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-7db1bc16-e859-4b4c-a0e8-6b9653dc824b tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Image pref 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 08 02:28:26.006580 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-7db1bc16-e859-4b4c-a0e8-6b9653dc824b tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 08 02:28:26.015471 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7db1bc16-e859-4b4c-a0e8-6b9653dc824b tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:26.015951 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7db1bc16-e859-4b4c-a0e8-6b9653dc824b tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:26.016420 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7db1bc16-e859-4b4c-a0e8-6b9653dc824b tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:26.017232 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7db1bc16-e859-4b4c-a0e8-6b9653dc824b tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:26.017735 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7db1bc16-e859-4b4c-a0e8-6b9653dc824b tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:26.018156 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7db1bc16-e859-4b4c-a0e8-6b9653dc824b tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:26.036005 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.quota [None req-7db1bc16-e859-4b4c-a0e8-6b9653dc824b tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Getting quotas for project 7203c52ae1f946bd9717a554316e4f4c. Resources: {'ram', 'cores', 'instances'} {{(pid=118352) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 08 02:28:26.040170 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.quota [None req-7db1bc16-e859-4b4c-a0e8-6b9653dc824b tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Getting quotas for user 5e3c891f2f3148f4aaba41a14d507cdf and project 7203c52ae1f946bd9717a554316e4f4c. Resources: {'ram', 'cores', 'instances'} {{(pid=118352) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 08 02:28:26.051847 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-7db1bc16-e859-4b4c-a0e8-6b9653dc824b tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Going to run 1 instances... {{(pid=118352) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Aug 08 02:28:26.054153 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-7db1bc16-e859-4b4c-a0e8-6b9653dc824b tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: f2edabed-f896-4ecb-ab34-ef137c01d581] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='30833f47-5987-41ca-b5ee-ad5ecafbb265',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118352) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Aug 08 02:28:26.207391 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-7db1bc16-e859-4b4c-a0e8-6b9653dc824b tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.709194 Aug 08 02:28:26.208659 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 191/384] 10.222.0.27 () {60 vars in 1147 bytes} [Mon Aug 8 02:28:25 2022] POST /compute/v2.1/servers => generated 380 bytes in 712 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 08 02:28:26.238157 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-aeb8c463-a5ca-402a-b4da-777d20d4b61b tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:26.239219 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-aeb8c463-a5ca-402a-b4da-777d20d4b61b tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: f2edabed-f896-4ecb-ab34-ef137c01d581] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:26.352261 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-aeb8c463-a5ca-402a-b4da-777d20d4b61b tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5e3c891f2f3148f4aaba41a14d507cdf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7203c52ae1f946bd9717a554316e4f4c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:26.362800 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-aeb8c463-a5ca-402a-b4da-777d20d4b61b tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581" status: 200 len: 1219 microversion: 2.1 time: 0.142639 Aug 08 02:28:26.363875 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 194/385] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:28:26 2022] GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581 => generated 1219 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:28:26.763568 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-9772e1b7-1a6d-422f-800f-8f043562de2b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:26.765573 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-9772e1b7-1a6d-422f-800f-8f043562de2b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:26.773125 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9772e1b7-1a6d-422f-800f-8f043562de2b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:26.773940 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9772e1b7-1a6d-422f-800f-8f043562de2b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:26.774768 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9772e1b7-1a6d-422f-800f-8f043562de2b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:26.775503 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9772e1b7-1a6d-422f-800f-8f043562de2b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:26.775955 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9772e1b7-1a6d-422f-800f-8f043562de2b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:26.776375 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9772e1b7-1a6d-422f-800f-8f043562de2b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:26.928020 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-9772e1b7-1a6d-422f-800f-8f043562de2b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:26.941784 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-9772e1b7-1a6d-422f-800f-8f043562de2b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:26.941784 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-9772e1b7-1a6d-422f-800f-8f043562de2b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:26.942302 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-9772e1b7-1a6d-422f-800f-8f043562de2b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1810 microversion: 2.20 time: 0.183200 Aug 08 02:28:26.943316 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 192/386] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:28:26 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1810 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 08 02:28:27.383242 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-da3ac51c-b792-4099-9617-f6da91042489 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:27.384136 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-da3ac51c-b792-4099-9617-f6da91042489 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: f2edabed-f896-4ecb-ab34-ef137c01d581] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:27.438111 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-da3ac51c-b792-4099-9617-f6da91042489 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5e3c891f2f3148f4aaba41a14d507cdf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7203c52ae1f946bd9717a554316e4f4c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:27.446002 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-da3ac51c-b792-4099-9617-f6da91042489 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581" status: 200 len: 1219 microversion: 2.1 time: 0.066551 Aug 08 02:28:27.446555 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 195/387] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:28:27 2022] GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581 => generated 1219 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:28:27.967733 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-ce50780d-c73b-496d-81b9-655571829676 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:27.969077 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-ce50780d-c73b-496d-81b9-655571829676 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:27.975825 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ce50780d-c73b-496d-81b9-655571829676 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:27.976306 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ce50780d-c73b-496d-81b9-655571829676 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:27.976907 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ce50780d-c73b-496d-81b9-655571829676 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:27.977659 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ce50780d-c73b-496d-81b9-655571829676 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:27.978363 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ce50780d-c73b-496d-81b9-655571829676 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:27.979048 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ce50780d-c73b-496d-81b9-655571829676 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:28.135677 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-ce50780d-c73b-496d-81b9-655571829676 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:28.147204 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-ce50780d-c73b-496d-81b9-655571829676 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:28.148731 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-ce50780d-c73b-496d-81b9-655571829676 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:28.149881 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-ce50780d-c73b-496d-81b9-655571829676 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1810 microversion: 2.20 time: 0.186460 Aug 08 02:28:28.150432 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 193/388] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:28:27 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1810 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:28:28.467864 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-a3f82695-e41d-47d1-ada7-ef5a2c441d57 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:28.469085 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-a3f82695-e41d-47d1-ada7-ef5a2c441d57 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: f2edabed-f896-4ecb-ab34-ef137c01d581] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:28.475964 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a3f82695-e41d-47d1-ada7-ef5a2c441d57 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:28.476689 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a3f82695-e41d-47d1-ada7-ef5a2c441d57 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:28.477300 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a3f82695-e41d-47d1-ada7-ef5a2c441d57 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:28.478152 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a3f82695-e41d-47d1-ada7-ef5a2c441d57 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:28.478810 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a3f82695-e41d-47d1-ada7-ef5a2c441d57 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:28.479397 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a3f82695-e41d-47d1-ada7-ef5a2c441d57 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:28.556688 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-a3f82695-e41d-47d1-ada7-ef5a2c441d57 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5e3c891f2f3148f4aaba41a14d507cdf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7203c52ae1f946bd9717a554316e4f4c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:28.567449 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-a3f82695-e41d-47d1-ada7-ef5a2c441d57 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581" status: 200 len: 1223 microversion: 2.1 time: 0.103652 Aug 08 02:28:28.568268 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 196/389] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:28:28 2022] GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581 => generated 1223 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:28:29.447268 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-ec6de6d4-8286-41f5-80d5-ce61fb5f6a08 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:29.447268 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-ec6de6d4-8286-41f5-80d5-ce61fb5f6a08 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:29.447268 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ec6de6d4-8286-41f5-80d5-ce61fb5f6a08 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:29.447268 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ec6de6d4-8286-41f5-80d5-ce61fb5f6a08 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:29.449023 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ec6de6d4-8286-41f5-80d5-ce61fb5f6a08 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:29.449023 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ec6de6d4-8286-41f5-80d5-ce61fb5f6a08 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:29.449023 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ec6de6d4-8286-41f5-80d5-ce61fb5f6a08 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:29.449023 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ec6de6d4-8286-41f5-80d5-ce61fb5f6a08 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:29.449757 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-ec6de6d4-8286-41f5-80d5-ce61fb5f6a08 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:29.449757 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-ec6de6d4-8286-41f5-80d5-ce61fb5f6a08 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:29.450369 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-ec6de6d4-8286-41f5-80d5-ce61fb5f6a08 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:29.450369 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-ec6de6d4-8286-41f5-80d5-ce61fb5f6a08 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1810 microversion: 2.20 time: 0.190562 Aug 08 02:28:29.450369 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 194/390] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:28:29 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1810 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 08 02:28:29.590544 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-093c1a71-0e01-4ee8-8531-29546ca46b57 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:29.590544 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-093c1a71-0e01-4ee8-8531-29546ca46b57 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: f2edabed-f896-4ecb-ab34-ef137c01d581] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:29.601761 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-093c1a71-0e01-4ee8-8531-29546ca46b57 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:29.603182 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-093c1a71-0e01-4ee8-8531-29546ca46b57 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:29.603182 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-093c1a71-0e01-4ee8-8531-29546ca46b57 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:29.603839 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-093c1a71-0e01-4ee8-8531-29546ca46b57 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:29.604298 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-093c1a71-0e01-4ee8-8531-29546ca46b57 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:29.604908 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-093c1a71-0e01-4ee8-8531-29546ca46b57 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:29.695101 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-093c1a71-0e01-4ee8-8531-29546ca46b57 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5e3c891f2f3148f4aaba41a14d507cdf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7203c52ae1f946bd9717a554316e4f4c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:29.705215 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-093c1a71-0e01-4ee8-8531-29546ca46b57 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581" status: 200 len: 1223 microversion: 2.1 time: 0.118927 Aug 08 02:28:29.705719 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 197/391] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:28:29 2022] GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581 => generated 1223 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:28:30.391257 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-86a471ec-b3b6-48f9-8ac4-87fc0405d003 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:30.391950 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-86a471ec-b3b6-48f9-8ac4-87fc0405d003 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:30.398604 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-86a471ec-b3b6-48f9-8ac4-87fc0405d003 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:30.399320 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-86a471ec-b3b6-48f9-8ac4-87fc0405d003 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:30.399994 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-86a471ec-b3b6-48f9-8ac4-87fc0405d003 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:30.400949 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-86a471ec-b3b6-48f9-8ac4-87fc0405d003 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:30.401493 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-86a471ec-b3b6-48f9-8ac4-87fc0405d003 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:30.401974 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-86a471ec-b3b6-48f9-8ac4-87fc0405d003 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:30.581179 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-86a471ec-b3b6-48f9-8ac4-87fc0405d003 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:30.581179 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-86a471ec-b3b6-48f9-8ac4-87fc0405d003 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:30.582039 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-86a471ec-b3b6-48f9-8ac4-87fc0405d003 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:30.582039 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-86a471ec-b3b6-48f9-8ac4-87fc0405d003 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1810 microversion: 2.20 time: 0.172849 Aug 08 02:28:30.582039 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 195/392] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:28:30 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1810 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:28:30.728788 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-fcc53e99-f334-4726-895e-a58ceebc0426 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:30.736791 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-fcc53e99-f334-4726-895e-a58ceebc0426 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: f2edabed-f896-4ecb-ab34-ef137c01d581] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:30.741776 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-fcc53e99-f334-4726-895e-a58ceebc0426 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:30.742310 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-fcc53e99-f334-4726-895e-a58ceebc0426 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:30.742778 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-fcc53e99-f334-4726-895e-a58ceebc0426 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:30.743571 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-fcc53e99-f334-4726-895e-a58ceebc0426 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:30.745184 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-fcc53e99-f334-4726-895e-a58ceebc0426 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:30.745184 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-fcc53e99-f334-4726-895e-a58ceebc0426 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:30.817021 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-fcc53e99-f334-4726-895e-a58ceebc0426 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5e3c891f2f3148f4aaba41a14d507cdf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7203c52ae1f946bd9717a554316e4f4c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:30.826202 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-fcc53e99-f334-4726-895e-a58ceebc0426 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581" status: 200 len: 1223 microversion: 2.1 time: 0.102404 Aug 08 02:28:30.826933 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 198/393] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:28:30 2022] GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581 => generated 1223 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:28:31.586402 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-2ecde8f9-2823-49d6-ba0b-a18ff5cae1c9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:31.587785 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-2ecde8f9-2823-49d6-ba0b-a18ff5cae1c9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:31.596792 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2ecde8f9-2823-49d6-ba0b-a18ff5cae1c9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:31.597572 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2ecde8f9-2823-49d6-ba0b-a18ff5cae1c9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:31.598250 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2ecde8f9-2823-49d6-ba0b-a18ff5cae1c9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:31.599341 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2ecde8f9-2823-49d6-ba0b-a18ff5cae1c9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:31.599784 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2ecde8f9-2823-49d6-ba0b-a18ff5cae1c9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:31.600271 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2ecde8f9-2823-49d6-ba0b-a18ff5cae1c9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:31.770279 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-2ecde8f9-2823-49d6-ba0b-a18ff5cae1c9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:31.781214 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-2ecde8f9-2823-49d6-ba0b-a18ff5cae1c9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:31.783086 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-2ecde8f9-2823-49d6-ba0b-a18ff5cae1c9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:31.784328 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-2ecde8f9-2823-49d6-ba0b-a18ff5cae1c9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1810 microversion: 2.20 time: 0.202252 Aug 08 02:28:31.793126 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 196/394] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:28:31 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1810 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:28:31.847595 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-81ca9a52-9c11-4de8-bd31-7d6625aa6ef3 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:31.848289 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-81ca9a52-9c11-4de8-bd31-7d6625aa6ef3 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: f2edabed-f896-4ecb-ab34-ef137c01d581] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:31.854628 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-81ca9a52-9c11-4de8-bd31-7d6625aa6ef3 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:31.855120 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-81ca9a52-9c11-4de8-bd31-7d6625aa6ef3 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:31.855559 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-81ca9a52-9c11-4de8-bd31-7d6625aa6ef3 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:31.856265 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-81ca9a52-9c11-4de8-bd31-7d6625aa6ef3 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:31.856750 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-81ca9a52-9c11-4de8-bd31-7d6625aa6ef3 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:31.857181 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-81ca9a52-9c11-4de8-bd31-7d6625aa6ef3 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:31.929326 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-81ca9a52-9c11-4de8-bd31-7d6625aa6ef3 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5e3c891f2f3148f4aaba41a14d507cdf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7203c52ae1f946bd9717a554316e4f4c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:31.939201 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-81ca9a52-9c11-4de8-bd31-7d6625aa6ef3 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581" status: 200 len: 1223 microversion: 2.1 time: 0.095589 Aug 08 02:28:31.939955 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 199/395] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:28:31 2022] GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581 => generated 1223 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:28:32.808779 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-40dac94c-794d-4bb0-8690-3020012470bf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:32.809520 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-40dac94c-794d-4bb0-8690-3020012470bf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:32.822050 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-40dac94c-794d-4bb0-8690-3020012470bf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:32.822793 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-40dac94c-794d-4bb0-8690-3020012470bf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:32.823331 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-40dac94c-794d-4bb0-8690-3020012470bf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:32.824179 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-40dac94c-794d-4bb0-8690-3020012470bf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:32.825292 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-40dac94c-794d-4bb0-8690-3020012470bf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:32.825823 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-40dac94c-794d-4bb0-8690-3020012470bf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:32.963447 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-e337dde6-e2f6-4bc4-8535-92bf42f4da53 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:32.971719 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-e337dde6-e2f6-4bc4-8535-92bf42f4da53 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: f2edabed-f896-4ecb-ab34-ef137c01d581] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:32.974440 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e337dde6-e2f6-4bc4-8535-92bf42f4da53 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:32.974953 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e337dde6-e2f6-4bc4-8535-92bf42f4da53 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:32.975379 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e337dde6-e2f6-4bc4-8535-92bf42f4da53 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:32.976082 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e337dde6-e2f6-4bc4-8535-92bf42f4da53 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:32.976531 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e337dde6-e2f6-4bc4-8535-92bf42f4da53 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:32.977004 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e337dde6-e2f6-4bc4-8535-92bf42f4da53 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:33.083530 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-e337dde6-e2f6-4bc4-8535-92bf42f4da53 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5e3c891f2f3148f4aaba41a14d507cdf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7203c52ae1f946bd9717a554316e4f4c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:33.087743 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-40dac94c-794d-4bb0-8690-3020012470bf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:33.092798 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-e337dde6-e2f6-4bc4-8535-92bf42f4da53 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581" status: 200 len: 1223 microversion: 2.1 time: 0.131516 Aug 08 02:28:33.095132 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 200/396] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:28:32 2022] GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581 => generated 1223 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:28:33.110083 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-40dac94c-794d-4bb0-8690-3020012470bf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:33.111715 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-40dac94c-794d-4bb0-8690-3020012470bf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:33.113669 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-40dac94c-794d-4bb0-8690-3020012470bf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1810 microversion: 2.20 time: 0.308756 Aug 08 02:28:33.114449 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 197/397] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:28:32 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1810 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:28:34.115745 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-19a346bd-6061-488f-8196-bf36a681a2cf tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:34.116388 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-19a346bd-6061-488f-8196-bf36a681a2cf tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: f2edabed-f896-4ecb-ab34-ef137c01d581] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:34.122839 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-19a346bd-6061-488f-8196-bf36a681a2cf tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:34.133221 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-19a346bd-6061-488f-8196-bf36a681a2cf tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:34.133221 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-19a346bd-6061-488f-8196-bf36a681a2cf tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:34.133221 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-19a346bd-6061-488f-8196-bf36a681a2cf tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:34.133221 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-19a346bd-6061-488f-8196-bf36a681a2cf tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:34.133551 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-19a346bd-6061-488f-8196-bf36a681a2cf tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:34.137612 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-1fb01b86-4776-4ae3-bb66-00d36528e98c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:34.138224 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-1fb01b86-4776-4ae3-bb66-00d36528e98c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:34.154998 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1fb01b86-4776-4ae3-bb66-00d36528e98c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:34.156076 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1fb01b86-4776-4ae3-bb66-00d36528e98c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:34.159085 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1fb01b86-4776-4ae3-bb66-00d36528e98c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:34.159828 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1fb01b86-4776-4ae3-bb66-00d36528e98c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:34.160315 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1fb01b86-4776-4ae3-bb66-00d36528e98c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:34.160760 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1fb01b86-4776-4ae3-bb66-00d36528e98c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:34.201867 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-19a346bd-6061-488f-8196-bf36a681a2cf tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5e3c891f2f3148f4aaba41a14d507cdf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7203c52ae1f946bd9717a554316e4f4c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:34.230342 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-19a346bd-6061-488f-8196-bf36a681a2cf tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581" status: 200 len: 1223 microversion: 2.1 time: 0.107552 Aug 08 02:28:34.230342 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 201/398] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:28:34 2022] GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581 => generated 1223 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:28:34.320761 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-1fb01b86-4776-4ae3-bb66-00d36528e98c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:34.339268 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-1fb01b86-4776-4ae3-bb66-00d36528e98c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:34.339268 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-1fb01b86-4776-4ae3-bb66-00d36528e98c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:34.339268 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-1fb01b86-4776-4ae3-bb66-00d36528e98c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1810 microversion: 2.20 time: 0.205408 Aug 08 02:28:34.339962 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 198/399] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:28:34 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1810 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 08 02:28:35.243006 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-01516f88-cfd9-4313-aa22-143c4b923a15 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:35.244532 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-01516f88-cfd9-4313-aa22-143c4b923a15 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: f2edabed-f896-4ecb-ab34-ef137c01d581] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:35.251556 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-01516f88-cfd9-4313-aa22-143c4b923a15 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:35.252379 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-01516f88-cfd9-4313-aa22-143c4b923a15 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:35.253147 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-01516f88-cfd9-4313-aa22-143c4b923a15 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:35.254416 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-01516f88-cfd9-4313-aa22-143c4b923a15 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:35.255345 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-01516f88-cfd9-4313-aa22-143c4b923a15 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:35.260039 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-01516f88-cfd9-4313-aa22-143c4b923a15 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:35.337182 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-01516f88-cfd9-4313-aa22-143c4b923a15 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5e3c891f2f3148f4aaba41a14d507cdf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7203c52ae1f946bd9717a554316e4f4c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:35.347196 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-01516f88-cfd9-4313-aa22-143c4b923a15 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581" status: 200 len: 1223 microversion: 2.1 time: 0.108795 Aug 08 02:28:35.348976 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 202/400] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:28:35 2022] GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581 => generated 1223 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:28:35.366158 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-ad3dff58-782f-425d-a38f-bfd1ae5fc214 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:35.367513 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-ad3dff58-782f-425d-a38f-bfd1ae5fc214 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:35.376608 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ad3dff58-782f-425d-a38f-bfd1ae5fc214 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:35.377445 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ad3dff58-782f-425d-a38f-bfd1ae5fc214 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:35.378085 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ad3dff58-782f-425d-a38f-bfd1ae5fc214 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:35.378952 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ad3dff58-782f-425d-a38f-bfd1ae5fc214 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:35.379581 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ad3dff58-782f-425d-a38f-bfd1ae5fc214 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:35.380363 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ad3dff58-782f-425d-a38f-bfd1ae5fc214 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:35.540238 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-ad3dff58-782f-425d-a38f-bfd1ae5fc214 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:35.553124 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-ad3dff58-782f-425d-a38f-bfd1ae5fc214 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:35.555573 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-ad3dff58-782f-425d-a38f-bfd1ae5fc214 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:35.556997 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-ad3dff58-782f-425d-a38f-bfd1ae5fc214 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1810 microversion: 2.20 time: 0.195004 Aug 08 02:28:35.557597 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 199/401] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:28:35 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1810 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 08 02:28:36.372904 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-8314cb4c-da39-4319-abf0-90174e4998a5 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:36.373584 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-8314cb4c-da39-4319-abf0-90174e4998a5 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: f2edabed-f896-4ecb-ab34-ef137c01d581] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:36.380471 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8314cb4c-da39-4319-abf0-90174e4998a5 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:36.380959 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8314cb4c-da39-4319-abf0-90174e4998a5 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:36.381389 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8314cb4c-da39-4319-abf0-90174e4998a5 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:36.382111 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8314cb4c-da39-4319-abf0-90174e4998a5 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:36.382558 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8314cb4c-da39-4319-abf0-90174e4998a5 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:36.383018 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8314cb4c-da39-4319-abf0-90174e4998a5 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:36.387396 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:36.454057 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-8314cb4c-da39-4319-abf0-90174e4998a5 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5e3c891f2f3148f4aaba41a14d507cdf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7203c52ae1f946bd9717a554316e4f4c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:36.463011 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-8314cb4c-da39-4319-abf0-90174e4998a5 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581" status: 200 len: 1223 microversion: 2.1 time: 0.094052 Aug 08 02:28:36.463578 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 203/402] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:28:36 2022] GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581 => generated 1223 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:28:36.587658 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-eb5e1cae-e991-4ccb-8073-00e550e4e889 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:36.588214 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-eb5e1cae-e991-4ccb-8073-00e550e4e889 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:36.590085 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell0:SELECT=2 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:36.590746 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell0:SELECT=6 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:36.596459 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-eb5e1cae-e991-4ccb-8073-00e550e4e889 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:36.596975 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-eb5e1cae-e991-4ccb-8073-00e550e4e889 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:36.597412 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-eb5e1cae-e991-4ccb-8073-00e550e4e889 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:36.598103 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-eb5e1cae-e991-4ccb-8073-00e550e4e889 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:36.598552 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-eb5e1cae-e991-4ccb-8073-00e550e4e889 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:36.598977 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-eb5e1cae-e991-4ccb-8073-00e550e4e889 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:36.729264 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-eb5e1cae-e991-4ccb-8073-00e550e4e889 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:36.740334 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-eb5e1cae-e991-4ccb-8073-00e550e4e889 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:36.741906 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-eb5e1cae-e991-4ccb-8073-00e550e4e889 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:36.743143 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-eb5e1cae-e991-4ccb-8073-00e550e4e889 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1810 microversion: 2.20 time: 0.159585 Aug 08 02:28:36.743761 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 200/403] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:28:36 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1810 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 08 02:28:37.483917 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-c62331c7-c5cc-496a-bbd2-3ab71bcb1389 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:37.484540 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-c62331c7-c5cc-496a-bbd2-3ab71bcb1389 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: f2edabed-f896-4ecb-ab34-ef137c01d581] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:37.487490 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell0:SELECT=4 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:37.493110 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c62331c7-c5cc-496a-bbd2-3ab71bcb1389 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:37.498221 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c62331c7-c5cc-496a-bbd2-3ab71bcb1389 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:37.498221 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c62331c7-c5cc-496a-bbd2-3ab71bcb1389 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:37.498221 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c62331c7-c5cc-496a-bbd2-3ab71bcb1389 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:37.498221 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c62331c7-c5cc-496a-bbd2-3ab71bcb1389 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:37.498554 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c62331c7-c5cc-496a-bbd2-3ab71bcb1389 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:37.602142 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-c62331c7-c5cc-496a-bbd2-3ab71bcb1389 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5e3c891f2f3148f4aaba41a14d507cdf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7203c52ae1f946bd9717a554316e4f4c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:37.609923 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-c62331c7-c5cc-496a-bbd2-3ab71bcb1389 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581" status: 200 len: 1215 microversion: 2.1 time: 0.130460 Aug 08 02:28:37.610487 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 204/404] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:28:37 2022] GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581 => generated 1215 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:28:37.771507 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-8d01a13d-1f38-4d57-b9e3-98cce46aaf7b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:37.772250 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-8d01a13d-1f38-4d57-b9e3-98cce46aaf7b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:37.780028 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8d01a13d-1f38-4d57-b9e3-98cce46aaf7b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:37.780599 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8d01a13d-1f38-4d57-b9e3-98cce46aaf7b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:37.781084 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8d01a13d-1f38-4d57-b9e3-98cce46aaf7b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:37.781761 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8d01a13d-1f38-4d57-b9e3-98cce46aaf7b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:37.782207 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8d01a13d-1f38-4d57-b9e3-98cce46aaf7b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:37.782790 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8d01a13d-1f38-4d57-b9e3-98cce46aaf7b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:37.955362 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-8d01a13d-1f38-4d57-b9e3-98cce46aaf7b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:37.964211 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-8d01a13d-1f38-4d57-b9e3-98cce46aaf7b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:37.965806 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-8d01a13d-1f38-4d57-b9e3-98cce46aaf7b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:37.966982 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-8d01a13d-1f38-4d57-b9e3-98cce46aaf7b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1810 microversion: 2.20 time: 0.199410 Aug 08 02:28:37.967538 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 201/405] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:28:37 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1810 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:28:38.632719 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-f01b7cfb-8bf8-4b6d-82a6-39ce496147d4 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:38.633334 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-f01b7cfb-8bf8-4b6d-82a6-39ce496147d4 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: f2edabed-f896-4ecb-ab34-ef137c01d581] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:38.639423 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f01b7cfb-8bf8-4b6d-82a6-39ce496147d4 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:38.640700 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f01b7cfb-8bf8-4b6d-82a6-39ce496147d4 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:38.641299 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f01b7cfb-8bf8-4b6d-82a6-39ce496147d4 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:38.642168 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f01b7cfb-8bf8-4b6d-82a6-39ce496147d4 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:38.642623 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f01b7cfb-8bf8-4b6d-82a6-39ce496147d4 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:38.643046 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f01b7cfb-8bf8-4b6d-82a6-39ce496147d4 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:38.716262 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-f01b7cfb-8bf8-4b6d-82a6-39ce496147d4 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5e3c891f2f3148f4aaba41a14d507cdf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7203c52ae1f946bd9717a554316e4f4c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:38.724368 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-f01b7cfb-8bf8-4b6d-82a6-39ce496147d4 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581" status: 200 len: 1279 microversion: 2.1 time: 0.097127 Aug 08 02:28:38.724964 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 205/406] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:28:38 2022] GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581 => generated 1279 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:28:38.992552 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-7c9a6d03-8d13-461b-918d-8c570e1ffd86 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:38.993303 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-7c9a6d03-8d13-461b-918d-8c570e1ffd86 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:39.001101 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7c9a6d03-8d13-461b-918d-8c570e1ffd86 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:39.001858 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7c9a6d03-8d13-461b-918d-8c570e1ffd86 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:39.002449 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7c9a6d03-8d13-461b-918d-8c570e1ffd86 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:39.003297 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7c9a6d03-8d13-461b-918d-8c570e1ffd86 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:39.003843 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7c9a6d03-8d13-461b-918d-8c570e1ffd86 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:39.004319 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7c9a6d03-8d13-461b-918d-8c570e1ffd86 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:39.180977 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-f2c07df8-4fd3-426d-93f9-d9b32eace471 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "7cdab395-b182-49c3-8bf5-3e6673f0dc52"}} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:28:39.193151 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-f2c07df8-4fd3-426d-93f9-d9b32eace471 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:39.199825 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f2c07df8-4fd3-426d-93f9-d9b32eace471 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:39.200751 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f2c07df8-4fd3-426d-93f9-d9b32eace471 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:39.201413 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f2c07df8-4fd3-426d-93f9-d9b32eace471 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:39.202686 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f2c07df8-4fd3-426d-93f9-d9b32eace471 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:39.203251 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f2c07df8-4fd3-426d-93f9-d9b32eace471 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:39.204129 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f2c07df8-4fd3-426d-93f9-d9b32eace471 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:39.388603 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-7c9a6d03-8d13-461b-918d-8c570e1ffd86 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:39.398381 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-7c9a6d03-8d13-461b-918d-8c570e1ffd86 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:39.402049 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-7c9a6d03-8d13-461b-918d-8c570e1ffd86 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:39.402049 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-7c9a6d03-8d13-461b-918d-8c570e1ffd86 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1810 microversion: 2.20 time: 0.413209 Aug 08 02:28:39.402049 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 202/407] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:28:38 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1810 bytes in 414 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:28:39.746378 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-c21073b2-e9d5-4e77-8f1f-d5f07aa1e1fa tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:39.746988 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-c21073b2-e9d5-4e77-8f1f-d5f07aa1e1fa tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: f2edabed-f896-4ecb-ab34-ef137c01d581] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:39.754207 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c21073b2-e9d5-4e77-8f1f-d5f07aa1e1fa tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:39.754956 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c21073b2-e9d5-4e77-8f1f-d5f07aa1e1fa tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:39.755650 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c21073b2-e9d5-4e77-8f1f-d5f07aa1e1fa tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:39.756413 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c21073b2-e9d5-4e77-8f1f-d5f07aa1e1fa tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:39.756944 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c21073b2-e9d5-4e77-8f1f-d5f07aa1e1fa tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:39.757417 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c21073b2-e9d5-4e77-8f1f-d5f07aa1e1fa tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:39.902490 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-c21073b2-e9d5-4e77-8f1f-d5f07aa1e1fa tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5e3c891f2f3148f4aaba41a14d507cdf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7203c52ae1f946bd9717a554316e4f4c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:39.911829 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-c21073b2-e9d5-4e77-8f1f-d5f07aa1e1fa tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581" status: 200 len: 1319 microversion: 2.1 time: 0.169630 Aug 08 02:28:39.931612 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 203/408] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:28:39 2022] GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581 => generated 1319 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:28:40.150085 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-f2c07df8-4fd3-426d-93f9-d9b32eace471 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "POST /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.973314 Aug 08 02:28:40.150688 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 206/409] 10.222.0.27 () {60 vars in 1323 bytes} [Mon Aug 8 02:28:39 2022] POST /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24/os-volume_attachments => generated 194 bytes in 975 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:28:40.429224 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-0b3abd80-3cab-4baa-aa20-c15f320ed356 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:40.430148 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-0b3abd80-3cab-4baa-aa20-c15f320ed356 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:40.436073 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0b3abd80-3cab-4baa-aa20-c15f320ed356 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:40.436641 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0b3abd80-3cab-4baa-aa20-c15f320ed356 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:40.437123 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0b3abd80-3cab-4baa-aa20-c15f320ed356 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:40.437809 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0b3abd80-3cab-4baa-aa20-c15f320ed356 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:40.438291 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0b3abd80-3cab-4baa-aa20-c15f320ed356 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:40.438730 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0b3abd80-3cab-4baa-aa20-c15f320ed356 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:40.624464 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-0b3abd80-3cab-4baa-aa20-c15f320ed356 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:40.633574 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-0b3abd80-3cab-4baa-aa20-c15f320ed356 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:40.635281 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-0b3abd80-3cab-4baa-aa20-c15f320ed356 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:40.636727 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-0b3abd80-3cab-4baa-aa20-c15f320ed356 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1810 microversion: 2.20 time: 0.215354 Aug 08 02:28:40.637295 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 204/410] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:28:40 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1810 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:28:40.731351 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [req-ecc1f7d6-b2f1-45d7-821a-43d909ab366f req-e266fa53-ec86-4864-be4d-d926a9398d31 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f2edabed-f896-4ecb-ab34-ef137c01d581", "tag": "f553eec7-8945-4f14-8a23-9cc399094640"}]} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:28:40.741496 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-ecc1f7d6-b2f1-45d7-821a-43d909ab366f req-e266fa53-ec86-4864-be4d-d926a9398d31 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:40.742007 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-ecc1f7d6-b2f1-45d7-821a-43d909ab366f req-e266fa53-ec86-4864-be4d-d926a9398d31 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:40.742448 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-ecc1f7d6-b2f1-45d7-821a-43d909ab366f req-e266fa53-ec86-4864-be4d-d926a9398d31 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:40.754383 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.compute.server_external_events [req-ecc1f7d6-b2f1-45d7-821a-43d909ab366f req-e266fa53-ec86-4864-be4d-d926a9398d31 service nova] Creating event network-changed:f553eec7-8945-4f14-8a23-9cc399094640 for instance f2edabed-f896-4ecb-ab34-ef137c01d581 on hv-cinder-74014 Aug 08 02:28:40.759102 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [req-ecc1f7d6-b2f1-45d7-821a-43d909ab366f req-e266fa53-ec86-4864-be4d-d926a9398d31 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034666 Aug 08 02:28:40.760035 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 207/411] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:28:40 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:28:40.941718 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-4efa9744-01d5-42d6-8d62-fac700a872d2 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:40.941718 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-4efa9744-01d5-42d6-8d62-fac700a872d2 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: f2edabed-f896-4ecb-ab34-ef137c01d581] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:40.948131 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4efa9744-01d5-42d6-8d62-fac700a872d2 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:40.948614 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4efa9744-01d5-42d6-8d62-fac700a872d2 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:40.949073 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4efa9744-01d5-42d6-8d62-fac700a872d2 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:40.949760 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4efa9744-01d5-42d6-8d62-fac700a872d2 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:40.950209 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4efa9744-01d5-42d6-8d62-fac700a872d2 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:40.950623 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4efa9744-01d5-42d6-8d62-fac700a872d2 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:40.970688 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=2 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:41.086024 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-4efa9744-01d5-42d6-8d62-fac700a872d2 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5e3c891f2f3148f4aaba41a14d507cdf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7203c52ae1f946bd9717a554316e4f4c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:41.097661 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-4efa9744-01d5-42d6-8d62-fac700a872d2 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581" status: 200 len: 1319 microversion: 2.1 time: 0.159301 Aug 08 02:28:41.097661 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 205/412] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:28:40 2022] GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581 => generated 1319 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:28:41.661888 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-8bac95f2-dba8-4733-b4ee-a4e3f446a7b7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:41.662609 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-8bac95f2-dba8-4733-b4ee-a4e3f446a7b7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:41.669068 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8bac95f2-dba8-4733-b4ee-a4e3f446a7b7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:41.669556 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8bac95f2-dba8-4733-b4ee-a4e3f446a7b7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:41.670023 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8bac95f2-dba8-4733-b4ee-a4e3f446a7b7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:41.671276 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8bac95f2-dba8-4733-b4ee-a4e3f446a7b7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:41.671803 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8bac95f2-dba8-4733-b4ee-a4e3f446a7b7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:41.672460 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8bac95f2-dba8-4733-b4ee-a4e3f446a7b7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:41.817288 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-8bac95f2-dba8-4733-b4ee-a4e3f446a7b7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:41.830293 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-8bac95f2-dba8-4733-b4ee-a4e3f446a7b7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:41.832010 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-8bac95f2-dba8-4733-b4ee-a4e3f446a7b7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:41.833526 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-8bac95f2-dba8-4733-b4ee-a4e3f446a7b7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1810 microversion: 2.20 time: 0.176084 Aug 08 02:28:41.834200 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 208/413] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:28:41 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1810 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:28:42.115747 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-6cd7e397-c4b1-4309-9d57-efdf4fb2f14d tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:42.116419 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-6cd7e397-c4b1-4309-9d57-efdf4fb2f14d tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: f2edabed-f896-4ecb-ab34-ef137c01d581] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:42.125403 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6cd7e397-c4b1-4309-9d57-efdf4fb2f14d tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:42.125403 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6cd7e397-c4b1-4309-9d57-efdf4fb2f14d tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:42.125701 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6cd7e397-c4b1-4309-9d57-efdf4fb2f14d tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:42.126351 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6cd7e397-c4b1-4309-9d57-efdf4fb2f14d tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:42.126811 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6cd7e397-c4b1-4309-9d57-efdf4fb2f14d tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:42.127246 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6cd7e397-c4b1-4309-9d57-efdf4fb2f14d tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:42.280866 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-6cd7e397-c4b1-4309-9d57-efdf4fb2f14d tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5e3c891f2f3148f4aaba41a14d507cdf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7203c52ae1f946bd9717a554316e4f4c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:42.288485 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-6cd7e397-c4b1-4309-9d57-efdf4fb2f14d tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581" status: 200 len: 1319 microversion: 2.1 time: 0.176617 Aug 08 02:28:42.291598 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 206/414] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:28:42 2022] GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581 => generated 1319 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:28:42.860304 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-5912a2d7-793c-4f23-b43a-3257ed972104 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:42.860973 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-5912a2d7-793c-4f23-b43a-3257ed972104 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:42.867838 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5912a2d7-793c-4f23-b43a-3257ed972104 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:42.868327 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5912a2d7-793c-4f23-b43a-3257ed972104 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:42.868795 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5912a2d7-793c-4f23-b43a-3257ed972104 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:42.869498 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5912a2d7-793c-4f23-b43a-3257ed972104 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:42.869952 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5912a2d7-793c-4f23-b43a-3257ed972104 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:42.870381 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5912a2d7-793c-4f23-b43a-3257ed972104 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:43.012855 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-5912a2d7-793c-4f23-b43a-3257ed972104 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:43.021089 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-5912a2d7-793c-4f23-b43a-3257ed972104 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:43.022607 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-5912a2d7-793c-4f23-b43a-3257ed972104 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:43.023768 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-5912a2d7-793c-4f23-b43a-3257ed972104 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1810 microversion: 2.20 time: 0.168059 Aug 08 02:28:43.024335 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 209/415] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:28:42 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1810 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:28:43.311266 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-4798ba33-7b31-4570-86d9-f0d6aa95a1d1 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:43.312897 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-4798ba33-7b31-4570-86d9-f0d6aa95a1d1 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: f2edabed-f896-4ecb-ab34-ef137c01d581] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:43.319886 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4798ba33-7b31-4570-86d9-f0d6aa95a1d1 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:43.320376 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4798ba33-7b31-4570-86d9-f0d6aa95a1d1 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:43.320880 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4798ba33-7b31-4570-86d9-f0d6aa95a1d1 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:43.321590 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4798ba33-7b31-4570-86d9-f0d6aa95a1d1 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:43.322049 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4798ba33-7b31-4570-86d9-f0d6aa95a1d1 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:43.322479 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4798ba33-7b31-4570-86d9-f0d6aa95a1d1 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:43.479480 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-4798ba33-7b31-4570-86d9-f0d6aa95a1d1 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5e3c891f2f3148f4aaba41a14d507cdf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7203c52ae1f946bd9717a554316e4f4c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:43.488884 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-4798ba33-7b31-4570-86d9-f0d6aa95a1d1 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581" status: 200 len: 1319 microversion: 2.1 time: 0.181874 Aug 08 02:28:43.490816 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 207/416] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:28:43 2022] GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581 => generated 1319 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:28:44.048846 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-2d679376-4f35-41d5-a19e-39fed63f79a2 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:44.049570 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-2d679376-4f35-41d5-a19e-39fed63f79a2 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:44.056094 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2d679376-4f35-41d5-a19e-39fed63f79a2 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:44.056572 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2d679376-4f35-41d5-a19e-39fed63f79a2 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:44.057242 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2d679376-4f35-41d5-a19e-39fed63f79a2 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:44.057935 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2d679376-4f35-41d5-a19e-39fed63f79a2 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:44.058379 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2d679376-4f35-41d5-a19e-39fed63f79a2 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:44.058799 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2d679376-4f35-41d5-a19e-39fed63f79a2 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:44.211937 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-2d679376-4f35-41d5-a19e-39fed63f79a2 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:44.222415 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-2d679376-4f35-41d5-a19e-39fed63f79a2 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:44.224083 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-2d679376-4f35-41d5-a19e-39fed63f79a2 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:44.225415 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-2d679376-4f35-41d5-a19e-39fed63f79a2 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1810 microversion: 2.20 time: 0.181820 Aug 08 02:28:44.226075 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 210/417] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:28:44 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1810 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:28:44.510612 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-19b6a655-44c0-4ba0-a8bd-79dad0dbb3bb tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:44.511830 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-19b6a655-44c0-4ba0-a8bd-79dad0dbb3bb tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: f2edabed-f896-4ecb-ab34-ef137c01d581] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:44.516294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_api:SELECT=57,nova_api:INSERT=3 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:44.519983 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-19b6a655-44c0-4ba0-a8bd-79dad0dbb3bb tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:44.520477 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-19b6a655-44c0-4ba0-a8bd-79dad0dbb3bb tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:44.527716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-19b6a655-44c0-4ba0-a8bd-79dad0dbb3bb tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:44.527716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-19b6a655-44c0-4ba0-a8bd-79dad0dbb3bb tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:44.527716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-19b6a655-44c0-4ba0-a8bd-79dad0dbb3bb tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:44.527716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-19b6a655-44c0-4ba0-a8bd-79dad0dbb3bb tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:44.721364 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-19b6a655-44c0-4ba0-a8bd-79dad0dbb3bb tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5e3c891f2f3148f4aaba41a14d507cdf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7203c52ae1f946bd9717a554316e4f4c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:44.730685 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-19b6a655-44c0-4ba0-a8bd-79dad0dbb3bb tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581" status: 200 len: 1319 microversion: 2.1 time: 0.223727 Aug 08 02:28:44.731124 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 208/418] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:28:44 2022] GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581 => generated 1319 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:28:45.250603 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-5b573133-cf8a-4230-b8d4-12d591adcbbe tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:45.251889 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-5b573133-cf8a-4230-b8d4-12d591adcbbe tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:45.258290 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5b573133-cf8a-4230-b8d4-12d591adcbbe tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:45.258779 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5b573133-cf8a-4230-b8d4-12d591adcbbe tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:45.259211 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5b573133-cf8a-4230-b8d4-12d591adcbbe tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:45.259916 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5b573133-cf8a-4230-b8d4-12d591adcbbe tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:45.260371 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5b573133-cf8a-4230-b8d4-12d591adcbbe tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:45.260906 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5b573133-cf8a-4230-b8d4-12d591adcbbe tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:45.455499 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-5b573133-cf8a-4230-b8d4-12d591adcbbe tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:45.474205 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-5b573133-cf8a-4230-b8d4-12d591adcbbe tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:45.478067 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-5b573133-cf8a-4230-b8d4-12d591adcbbe tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:45.482012 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-5b573133-cf8a-4230-b8d4-12d591adcbbe tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1810 microversion: 2.20 time: 0.235046 Aug 08 02:28:45.485499 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 211/419] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:28:45 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1810 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:28:45.752467 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-e1ed2595-3eec-4206-88d3-13914adacc5b tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:45.757594 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-e1ed2595-3eec-4206-88d3-13914adacc5b tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: f2edabed-f896-4ecb-ab34-ef137c01d581] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:45.760437 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e1ed2595-3eec-4206-88d3-13914adacc5b tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:45.761147 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e1ed2595-3eec-4206-88d3-13914adacc5b tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:45.761768 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e1ed2595-3eec-4206-88d3-13914adacc5b tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:45.762485 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e1ed2595-3eec-4206-88d3-13914adacc5b tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:45.762945 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e1ed2595-3eec-4206-88d3-13914adacc5b tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:45.763370 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e1ed2595-3eec-4206-88d3-13914adacc5b tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:45.918658 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-e1ed2595-3eec-4206-88d3-13914adacc5b tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5e3c891f2f3148f4aaba41a14d507cdf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7203c52ae1f946bd9717a554316e4f4c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:45.931754 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-e1ed2595-3eec-4206-88d3-13914adacc5b tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581" status: 200 len: 1319 microversion: 2.1 time: 0.184120 Aug 08 02:28:45.932636 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 209/420] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:28:45 2022] GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581 => generated 1319 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:28:46.082098 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [req-0752d67e-3238-447e-83a5-9b9f2db8c6c0 req-45b86313-5729-4012-ac00-d21086f35066 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f2edabed-f896-4ecb-ab34-ef137c01d581", "name": "network-vif-plugged", "status": "completed", "tag": "f553eec7-8945-4f14-8a23-9cc399094640"}]} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:28:46.093917 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_api:SELECT=56,nova_api:UPDATE=1 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:46.099890 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-0752d67e-3238-447e-83a5-9b9f2db8c6c0 req-45b86313-5729-4012-ac00-d21086f35066 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:46.100372 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-0752d67e-3238-447e-83a5-9b9f2db8c6c0 req-45b86313-5729-4012-ac00-d21086f35066 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:46.100831 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-0752d67e-3238-447e-83a5-9b9f2db8c6c0 req-45b86313-5729-4012-ac00-d21086f35066 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:46.114450 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.compute.server_external_events [req-0752d67e-3238-447e-83a5-9b9f2db8c6c0 req-45b86313-5729-4012-ac00-d21086f35066 service nova] Creating event network-vif-plugged:f553eec7-8945-4f14-8a23-9cc399094640 for instance f2edabed-f896-4ecb-ab34-ef137c01d581 on hv-cinder-74014 Aug 08 02:28:46.119312 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [req-0752d67e-3238-447e-83a5-9b9f2db8c6c0 req-45b86313-5729-4012-ac00-d21086f35066 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041921 Aug 08 02:28:46.120362 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 212/421] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:28:46 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:28:46.520579 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-7ee47c93-60a2-4cb7-957d-a31b547a1c8d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:46.525557 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-7ee47c93-60a2-4cb7-957d-a31b547a1c8d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:46.528160 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7ee47c93-60a2-4cb7-957d-a31b547a1c8d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:46.528697 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7ee47c93-60a2-4cb7-957d-a31b547a1c8d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:46.529324 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7ee47c93-60a2-4cb7-957d-a31b547a1c8d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:46.530273 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7ee47c93-60a2-4cb7-957d-a31b547a1c8d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:46.530752 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7ee47c93-60a2-4cb7-957d-a31b547a1c8d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:46.531347 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7ee47c93-60a2-4cb7-957d-a31b547a1c8d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:46.703434 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-7ee47c93-60a2-4cb7-957d-a31b547a1c8d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:46.713304 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-7ee47c93-60a2-4cb7-957d-a31b547a1c8d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:46.714817 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-7ee47c93-60a2-4cb7-957d-a31b547a1c8d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:46.716146 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-7ee47c93-60a2-4cb7-957d-a31b547a1c8d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1810 microversion: 2.20 time: 0.200246 Aug 08 02:28:46.716725 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 210/422] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:28:46 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1810 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:28:46.954582 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-27704343-3f39-4a2e-ab6a-87373598d3e8 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:46.955334 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-27704343-3f39-4a2e-ab6a-87373598d3e8 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: f2edabed-f896-4ecb-ab34-ef137c01d581] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:46.973230 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-27704343-3f39-4a2e-ab6a-87373598d3e8 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:46.973794 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-27704343-3f39-4a2e-ab6a-87373598d3e8 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:46.974171 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-27704343-3f39-4a2e-ab6a-87373598d3e8 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:46.974890 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-27704343-3f39-4a2e-ab6a-87373598d3e8 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:46.976089 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-27704343-3f39-4a2e-ab6a-87373598d3e8 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:46.976473 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-27704343-3f39-4a2e-ab6a-87373598d3e8 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:47.159591 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-27704343-3f39-4a2e-ab6a-87373598d3e8 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5e3c891f2f3148f4aaba41a14d507cdf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7203c52ae1f946bd9717a554316e4f4c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:47.169302 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-27704343-3f39-4a2e-ab6a-87373598d3e8 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581" status: 200 len: 1319 microversion: 2.1 time: 0.219363 Aug 08 02:28:47.170140 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 213/423] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:28:46 2022] GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581 => generated 1319 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:28:47.742128 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-38ab5127-f51f-4c97-8e66-1665274bacb5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:47.743022 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-38ab5127-f51f-4c97-8e66-1665274bacb5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:47.752812 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-38ab5127-f51f-4c97-8e66-1665274bacb5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:47.753838 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-38ab5127-f51f-4c97-8e66-1665274bacb5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:47.754740 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-38ab5127-f51f-4c97-8e66-1665274bacb5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:47.767568 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-38ab5127-f51f-4c97-8e66-1665274bacb5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:47.768174 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-38ab5127-f51f-4c97-8e66-1665274bacb5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:47.768693 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-38ab5127-f51f-4c97-8e66-1665274bacb5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:47.992842 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-38ab5127-f51f-4c97-8e66-1665274bacb5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:48.003405 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-38ab5127-f51f-4c97-8e66-1665274bacb5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:48.005167 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-38ab5127-f51f-4c97-8e66-1665274bacb5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:48.006520 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-38ab5127-f51f-4c97-8e66-1665274bacb5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1810 microversion: 2.20 time: 0.268566 Aug 08 02:28:48.007551 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 211/424] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:28:47 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1810 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:28:48.192599 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-c0edd848-47d7-4be1-b3ff-318652529a90 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:48.193954 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-c0edd848-47d7-4be1-b3ff-318652529a90 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: f2edabed-f896-4ecb-ab34-ef137c01d581] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:48.209476 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c0edd848-47d7-4be1-b3ff-318652529a90 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:48.210292 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c0edd848-47d7-4be1-b3ff-318652529a90 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:48.210888 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c0edd848-47d7-4be1-b3ff-318652529a90 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:48.211995 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c0edd848-47d7-4be1-b3ff-318652529a90 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:48.212705 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c0edd848-47d7-4be1-b3ff-318652529a90 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:48.213371 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c0edd848-47d7-4be1-b3ff-318652529a90 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:48.245627 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [req-1a2dbf23-8bd5-4e06-bedb-6d443067bdfa req-45189a37-db88-43a2-b7a5-5b93934acc0c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f51a5900-5abc-4188-bbc4-33b0f0f7d091", "name": "network-vif-plugged", "status": "completed", "tag": "8d2a6aac-ff75-417c-b0b9-cbf799f88d22"}]} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:28:48.266936 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-1a2dbf23-8bd5-4e06-bedb-6d443067bdfa req-45189a37-db88-43a2-b7a5-5b93934acc0c service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:48.267464 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-1a2dbf23-8bd5-4e06-bedb-6d443067bdfa req-45189a37-db88-43a2-b7a5-5b93934acc0c service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:48.267912 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-1a2dbf23-8bd5-4e06-bedb-6d443067bdfa req-45189a37-db88-43a2-b7a5-5b93934acc0c service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:48.286400 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.compute.server_external_events [req-1a2dbf23-8bd5-4e06-bedb-6d443067bdfa req-45189a37-db88-43a2-b7a5-5b93934acc0c service nova] Creating event network-vif-plugged:8d2a6aac-ff75-417c-b0b9-cbf799f88d22 for instance f51a5900-5abc-4188-bbc4-33b0f0f7d091 on hv-cinder-74014 Aug 08 02:28:48.291364 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [req-1a2dbf23-8bd5-4e06-bedb-6d443067bdfa req-45189a37-db88-43a2-b7a5-5b93934acc0c service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.052639 Aug 08 02:28:48.292307 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 212/425] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:28:48 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 08 02:28:48.400047 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-c0edd848-47d7-4be1-b3ff-318652529a90 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5e3c891f2f3148f4aaba41a14d507cdf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7203c52ae1f946bd9717a554316e4f4c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:48.410767 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-c0edd848-47d7-4be1-b3ff-318652529a90 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581" status: 200 len: 1319 microversion: 2.1 time: 0.221973 Aug 08 02:28:48.411355 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 214/426] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:28:48 2022] GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581 => generated 1319 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:28:48.877885 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-088d4262-e6ca-4be5-851a-710a6d8f1618 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:28:48.892947 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-088d4262-e6ca-4be5-851a-710a6d8f1618 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:48.900456 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-088d4262-e6ca-4be5-851a-710a6d8f1618 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:48.900998 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-088d4262-e6ca-4be5-851a-710a6d8f1618 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:48.901503 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-088d4262-e6ca-4be5-851a-710a6d8f1618 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:48.902276 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-088d4262-e6ca-4be5-851a-710a6d8f1618 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:48.902971 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-088d4262-e6ca-4be5-851a-710a6d8f1618 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:48.903495 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-088d4262-e6ca-4be5-851a-710a6d8f1618 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:48.930351 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.objects.instance [None req-088d4262-e6ca-4be5-851a-710a6d8f1618 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lazy-loading 'flavor' on Instance uuid a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 {{(pid=118352) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 08 02:28:49.037978 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-809fb2a2-5f14-4d8e-b235-c023caf030c3 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:49.038601 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-809fb2a2-5f14-4d8e-b235-c023caf030c3 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:49.048286 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-809fb2a2-5f14-4d8e-b235-c023caf030c3 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:49.048824 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-809fb2a2-5f14-4d8e-b235-c023caf030c3 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:49.049261 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-809fb2a2-5f14-4d8e-b235-c023caf030c3 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:49.050280 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-809fb2a2-5f14-4d8e-b235-c023caf030c3 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:49.050895 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-809fb2a2-5f14-4d8e-b235-c023caf030c3 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:49.051752 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-809fb2a2-5f14-4d8e-b235-c023caf030c3 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:49.235576 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-809fb2a2-5f14-4d8e-b235-c023caf030c3 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:49.247346 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-809fb2a2-5f14-4d8e-b235-c023caf030c3 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:49.249698 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-809fb2a2-5f14-4d8e-b235-c023caf030c3 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:49.251071 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-809fb2a2-5f14-4d8e-b235-c023caf030c3 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1810 microversion: 2.20 time: 0.223912 Aug 08 02:28:49.253393 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 215/427] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:28:49 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1810 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 08 02:28:49.378838 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-088d4262-e6ca-4be5-851a-710a6d8f1618 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "POST /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24/action" status: 200 len: 32 microversion: 2.1 time: 0.507931 Aug 08 02:28:49.379447 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 213/428] 10.222.0.27 () {60 vars in 1278 bytes} [Mon Aug 8 02:28:48 2022] POST /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24/action => generated 32 bytes in 510 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Aug 08 02:28:49.389600 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-50601560-6249-4d9c-8d12-df3d5b01017b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:49.390846 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-50601560-6249-4d9c-8d12-df3d5b01017b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:49.398179 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-50601560-6249-4d9c-8d12-df3d5b01017b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:49.398794 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-50601560-6249-4d9c-8d12-df3d5b01017b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:49.403064 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-50601560-6249-4d9c-8d12-df3d5b01017b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:49.403064 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-50601560-6249-4d9c-8d12-df3d5b01017b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:49.403064 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-50601560-6249-4d9c-8d12-df3d5b01017b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:49.403064 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-50601560-6249-4d9c-8d12-df3d5b01017b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:49.437608 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-f228d61b-3a27-42a0-ac79-569879438210 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:49.437608 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-f228d61b-3a27-42a0-ac79-569879438210 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: f2edabed-f896-4ecb-ab34-ef137c01d581] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:49.451640 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f228d61b-3a27-42a0-ac79-569879438210 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:49.452756 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f228d61b-3a27-42a0-ac79-569879438210 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:49.453186 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f228d61b-3a27-42a0-ac79-569879438210 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:49.454086 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f228d61b-3a27-42a0-ac79-569879438210 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:49.454721 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f228d61b-3a27-42a0-ac79-569879438210 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:49.455751 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f228d61b-3a27-42a0-ac79-569879438210 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:49.628134 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-50601560-6249-4d9c-8d12-df3d5b01017b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:49.637441 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-50601560-6249-4d9c-8d12-df3d5b01017b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1736 microversion: 2.1 time: 0.252320 Aug 08 02:28:49.638127 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 216/429] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:28:49 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1736 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:28:49.647477 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-f228d61b-3a27-42a0-ac79-569879438210 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5e3c891f2f3148f4aaba41a14d507cdf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7203c52ae1f946bd9717a554316e4f4c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:49.655743 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-f228d61b-3a27-42a0-ac79-569879438210 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581" status: 200 len: 1508 microversion: 2.1 time: 0.226685 Aug 08 02:28:49.656317 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 214/430] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:28:49 2022] GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581 => generated 1508 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:28:50.060333 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-59e0b5f2-b14d-4a25-80fe-912c266a7398 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:50.060891 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-59e0b5f2-b14d-4a25-80fe-912c266a7398 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:50.067522 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-59e0b5f2-b14d-4a25-80fe-912c266a7398 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:50.068014 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-59e0b5f2-b14d-4a25-80fe-912c266a7398 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:50.068454 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-59e0b5f2-b14d-4a25-80fe-912c266a7398 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:50.069179 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-59e0b5f2-b14d-4a25-80fe-912c266a7398 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:50.069635 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-59e0b5f2-b14d-4a25-80fe-912c266a7398 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:50.075517 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-59e0b5f2-b14d-4a25-80fe-912c266a7398 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:50.221829 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.objects.instance [None req-59e0b5f2-b14d-4a25-80fe-912c266a7398 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lazy-loading 'services' on Instance uuid ddd15bdd-8a1c-455b-91c9-504f69b5308a {{(pid=118351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 08 02:28:50.277981 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-c62b0a87-64f2-4f97-9aab-f397e4372585 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:50.279201 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-c62b0a87-64f2-4f97-9aab-f397e4372585 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:50.294274 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c62b0a87-64f2-4f97-9aab-f397e4372585 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:50.294777 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c62b0a87-64f2-4f97-9aab-f397e4372585 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:50.295220 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c62b0a87-64f2-4f97-9aab-f397e4372585 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:50.296827 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c62b0a87-64f2-4f97-9aab-f397e4372585 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:50.297307 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c62b0a87-64f2-4f97-9aab-f397e4372585 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:50.297744 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c62b0a87-64f2-4f97-9aab-f397e4372585 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:50.462749 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-c62b0a87-64f2-4f97-9aab-f397e4372585 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:50.474174 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-c62b0a87-64f2-4f97-9aab-f397e4372585 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:50.509128 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-c62b0a87-64f2-4f97-9aab-f397e4372585 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:50.509128 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-c62b0a87-64f2-4f97-9aab-f397e4372585 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1810 microversion: 2.20 time: 0.216429 Aug 08 02:28:50.509128 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 215/431] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:28:50 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1810 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 08 02:28:50.509731 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-59e0b5f2-b14d-4a25-80fe-912c266a7398 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "DELETE /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a/os-volume_attachments/a4ca4896-7117-4711-b088-f45bc051b0d4" status: 202 len: 0 microversion: 2.1 time: 0.436898 Aug 08 02:28:50.509731 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 217/432] 10.222.0.27 () {58 vars in 1416 bytes} [Mon Aug 8 02:28:50 2022] DELETE /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a/os-volume_attachments/a4ca4896-7117-4711-b088-f45bc051b0d4 => generated 0 bytes in 438 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 08 02:28:50.659444 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-dc82e34c-79e0-4ea8-951e-7cd1dbffdaee tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:50.660043 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-dc82e34c-79e0-4ea8-951e-7cd1dbffdaee tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:50.666329 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dc82e34c-79e0-4ea8-951e-7cd1dbffdaee tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:50.666813 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dc82e34c-79e0-4ea8-951e-7cd1dbffdaee tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:50.667249 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dc82e34c-79e0-4ea8-951e-7cd1dbffdaee tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:50.667956 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dc82e34c-79e0-4ea8-951e-7cd1dbffdaee tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:50.668402 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dc82e34c-79e0-4ea8-951e-7cd1dbffdaee tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:50.668859 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dc82e34c-79e0-4ea8-951e-7cd1dbffdaee tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:50.833668 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-dc82e34c-79e0-4ea8-951e-7cd1dbffdaee tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:50.848833 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-dc82e34c-79e0-4ea8-951e-7cd1dbffdaee tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1736 microversion: 2.1 time: 0.193808 Aug 08 02:28:50.849642 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 216/433] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:28:50 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1736 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:28:51.518040 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-6d4b8730-4b85-4506-b2e1-ad5afd01000d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:51.520067 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-6d4b8730-4b85-4506-b2e1-ad5afd01000d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:51.530706 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6d4b8730-4b85-4506-b2e1-ad5afd01000d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:51.531662 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6d4b8730-4b85-4506-b2e1-ad5afd01000d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:51.532368 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6d4b8730-4b85-4506-b2e1-ad5afd01000d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:51.533740 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6d4b8730-4b85-4506-b2e1-ad5afd01000d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:51.534477 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6d4b8730-4b85-4506-b2e1-ad5afd01000d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:51.535146 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6d4b8730-4b85-4506-b2e1-ad5afd01000d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:51.753675 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-6d4b8730-4b85-4506-b2e1-ad5afd01000d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:51.763053 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-6d4b8730-4b85-4506-b2e1-ad5afd01000d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:51.773306 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-6d4b8730-4b85-4506-b2e1-ad5afd01000d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:51.775861 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-6d4b8730-4b85-4506-b2e1-ad5afd01000d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1810 microversion: 2.20 time: 0.263252 Aug 08 02:28:51.777391 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 218/434] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:28:51 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1810 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:28:51.870828 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-4bc11374-cf32-4ec0-8ab9-1d8a48da6ab3 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:51.872658 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-4bc11374-cf32-4ec0-8ab9-1d8a48da6ab3 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:51.883032 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4bc11374-cf32-4ec0-8ab9-1d8a48da6ab3 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:51.883032 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4bc11374-cf32-4ec0-8ab9-1d8a48da6ab3 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:51.883032 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4bc11374-cf32-4ec0-8ab9-1d8a48da6ab3 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:51.883389 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4bc11374-cf32-4ec0-8ab9-1d8a48da6ab3 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:51.883858 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4bc11374-cf32-4ec0-8ab9-1d8a48da6ab3 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:51.884398 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4bc11374-cf32-4ec0-8ab9-1d8a48da6ab3 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:52.107936 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-4bc11374-cf32-4ec0-8ab9-1d8a48da6ab3 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:52.118740 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-4bc11374-cf32-4ec0-8ab9-1d8a48da6ab3 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1736 microversion: 2.1 time: 0.252159 Aug 08 02:28:52.120240 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 217/435] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:28:51 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1736 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:28:52.812525 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-7d33120e-6867-42ea-aa5b-dff06c6d0bf6 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:52.813189 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-7d33120e-6867-42ea-aa5b-dff06c6d0bf6 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:52.826593 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7d33120e-6867-42ea-aa5b-dff06c6d0bf6 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:52.827617 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7d33120e-6867-42ea-aa5b-dff06c6d0bf6 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:52.832276 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7d33120e-6867-42ea-aa5b-dff06c6d0bf6 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:52.834269 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7d33120e-6867-42ea-aa5b-dff06c6d0bf6 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:52.834761 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7d33120e-6867-42ea-aa5b-dff06c6d0bf6 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:52.835304 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7d33120e-6867-42ea-aa5b-dff06c6d0bf6 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:52.877666 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:53.091294 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-7d33120e-6867-42ea-aa5b-dff06c6d0bf6 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:53.116272 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-7d33120e-6867-42ea-aa5b-dff06c6d0bf6 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:53.118451 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-7d33120e-6867-42ea-aa5b-dff06c6d0bf6 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:53.120176 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-7d33120e-6867-42ea-aa5b-dff06c6d0bf6 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1810 microversion: 2.20 time: 0.325051 Aug 08 02:28:53.120765 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 219/436] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:28:52 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1810 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:28:53.144101 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-c74e1a18-0ab5-4881-96c4-09a1e08895f4 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:53.144862 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-c74e1a18-0ab5-4881-96c4-09a1e08895f4 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:53.151940 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c74e1a18-0ab5-4881-96c4-09a1e08895f4 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:53.152426 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c74e1a18-0ab5-4881-96c4-09a1e08895f4 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:53.152943 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c74e1a18-0ab5-4881-96c4-09a1e08895f4 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:53.153823 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c74e1a18-0ab5-4881-96c4-09a1e08895f4 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:53.154296 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c74e1a18-0ab5-4881-96c4-09a1e08895f4 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:53.154715 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c74e1a18-0ab5-4881-96c4-09a1e08895f4 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:53.451167 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-c74e1a18-0ab5-4881-96c4-09a1e08895f4 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:53.469820 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-c74e1a18-0ab5-4881-96c4-09a1e08895f4 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1736 microversion: 2.1 time: 0.327194 Aug 08 02:28:53.470464 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 218/437] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:28:53 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1736 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:28:53.865027 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-fa059f76-28a7-4ea5-b4b4-f4860b2ad85a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:28:53.885087 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-fa059f76-28a7-4ea5-b4b4-f4860b2ad85a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:53.885847 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-fa059f76-28a7-4ea5-b4b4-f4860b2ad85a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:53.886561 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-fa059f76-28a7-4ea5-b4b4-f4860b2ad85a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:53.959471 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-fa059f76-28a7-4ea5-b4b4-f4860b2ad85a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Going to try to stop instance {{(pid=118351) force_stop /opt/stack/nova/nova/compute/api.py:2709}} Aug 08 02:28:54.145237 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-7715661f-68c3-43fa-8445-c03e12240f97 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:54.146951 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-7715661f-68c3-43fa-8445-c03e12240f97 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:54.156885 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7715661f-68c3-43fa-8445-c03e12240f97 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:54.157677 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7715661f-68c3-43fa-8445-c03e12240f97 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:54.158300 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7715661f-68c3-43fa-8445-c03e12240f97 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:54.159239 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7715661f-68c3-43fa-8445-c03e12240f97 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:54.160563 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7715661f-68c3-43fa-8445-c03e12240f97 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:54.162817 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7715661f-68c3-43fa-8445-c03e12240f97 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:54.163486 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-fa059f76-28a7-4ea5-b4b4-f4860b2ad85a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "POST /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a/action" status: 202 len: 0 microversion: 2.1 time: 0.301461 Aug 08 02:28:54.164399 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 220/438] 10.222.0.27 () {60 vars in 1278 bytes} [Mon Aug 8 02:28:53 2022] POST /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a/action => generated 0 bytes in 305 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 08 02:28:54.177927 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-097d3f2d-4895-4e25-9da0-e786d31199c9 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:54.197088 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-097d3f2d-4895-4e25-9da0-e786d31199c9 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:54.197088 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-097d3f2d-4895-4e25-9da0-e786d31199c9 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:54.197088 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-097d3f2d-4895-4e25-9da0-e786d31199c9 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:54.197088 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-097d3f2d-4895-4e25-9da0-e786d31199c9 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:54.197488 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-097d3f2d-4895-4e25-9da0-e786d31199c9 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:54.197488 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-097d3f2d-4895-4e25-9da0-e786d31199c9 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:54.197488 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-097d3f2d-4895-4e25-9da0-e786d31199c9 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:54.423307 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-7715661f-68c3-43fa-8445-c03e12240f97 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:54.441959 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-7715661f-68c3-43fa-8445-c03e12240f97 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:54.443933 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-7715661f-68c3-43fa-8445-c03e12240f97 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:54.445636 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-7715661f-68c3-43fa-8445-c03e12240f97 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1797 microversion: 2.20 time: 0.305193 Aug 08 02:28:54.446766 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 219/439] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:28:54 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1797 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:28:54.497465 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-7721dd11-87c6-45b1-92ab-558827a30682 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:54.498525 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-7721dd11-87c6-45b1-92ab-558827a30682 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:54.502814 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-097d3f2d-4895-4e25-9da0-e786d31199c9 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:54.525974 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-097d3f2d-4895-4e25-9da0-e786d31199c9 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1680 microversion: 2.1 time: 0.356481 Aug 08 02:28:54.534832 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7721dd11-87c6-45b1-92ab-558827a30682 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:54.539748 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7721dd11-87c6-45b1-92ab-558827a30682 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:54.540455 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7721dd11-87c6-45b1-92ab-558827a30682 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:54.541690 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 221/440] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:28:54 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1680 bytes in 373 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:28:54.542098 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7721dd11-87c6-45b1-92ab-558827a30682 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:54.545740 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7721dd11-87c6-45b1-92ab-558827a30682 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:54.547093 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7721dd11-87c6-45b1-92ab-558827a30682 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:54.719164 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-7721dd11-87c6-45b1-92ab-558827a30682 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:54.722835 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:54.727792 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-7721dd11-87c6-45b1-92ab-558827a30682 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1736 microversion: 2.1 time: 0.234263 Aug 08 02:28:54.728360 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 220/441] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:28:54 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1736 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:28:55.144830 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-3f5aeded-fbc0-408e-98ab-1072289128f4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:55.145591 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-3f5aeded-fbc0-408e-98ab-1072289128f4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: dda0c0b0-ed8e-4d43-9b26-abe1bca05a61] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:55.173362 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3f5aeded-fbc0-408e-98ab-1072289128f4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:55.173914 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3f5aeded-fbc0-408e-98ab-1072289128f4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:55.179450 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3f5aeded-fbc0-408e-98ab-1072289128f4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:55.179450 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3f5aeded-fbc0-408e-98ab-1072289128f4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:55.179450 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3f5aeded-fbc0-408e-98ab-1072289128f4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:55.179450 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3f5aeded-fbc0-408e-98ab-1072289128f4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:55.204065 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-3f5aeded-fbc0-408e-98ab-1072289128f4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61/ips" status: 200 len: 151 microversion: 2.1 time: 0.073952 Aug 08 02:28:55.211162 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 222/442] 10.222.0.27 () {58 vars in 1248 bytes} [Mon Aug 8 02:28:55 2022] GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61/ips => generated 151 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:28:55.220077 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-403a6362-8ed6-4281-b198-3f2c662a70e7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "bacd7028-1966-45e9-bc6e-a6156c1d3960"}} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:28:55.225149 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-403a6362-8ed6-4281-b198-3f2c662a70e7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: dda0c0b0-ed8e-4d43-9b26-abe1bca05a61] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:55.236383 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-403a6362-8ed6-4281-b198-3f2c662a70e7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:55.236383 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-403a6362-8ed6-4281-b198-3f2c662a70e7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:55.236383 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-403a6362-8ed6-4281-b198-3f2c662a70e7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:55.236383 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-403a6362-8ed6-4281-b198-3f2c662a70e7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:55.236965 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-403a6362-8ed6-4281-b198-3f2c662a70e7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:55.236965 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-403a6362-8ed6-4281-b198-3f2c662a70e7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:55.354813 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.wsgi [None req-403a6362-8ed6-4281-b198-3f2c662a70e7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] HTTP exception thrown: Invalid volume: volume bacd7028-1966-45e9-bc6e-a6156c1d3960 is already attached to instances: 95ae6b45-35d7-43a7-a021-91bb545cac4e Aug 08 02:28:55.357558 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-403a6362-8ed6-4281-b198-3f2c662a70e7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Request method failure captured: Aug 08 02:28:55.357558 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: request: POST /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61/os-volume_attachments HTTP/1.1 Aug 08 02:28:55.357558 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept: application/json Aug 08 02:28:55.357558 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept-Encoding: identity Aug 08 02:28:55.357558 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Connection: close Aug 08 02:28:55.357558 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Length: 74 Aug 08 02:28:55.357558 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Type: application/json Aug 08 02:28:55.357558 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Host: 10.222.0.27 Aug 08 02:28:55.357558 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Openstack-System-Scope: None Aug 08 02:28:55.357558 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: User-Agent: python-urllib3/1.26.11 Aug 08 02:28:55.357558 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Auth-Token: gAAAAABi8HQUs7QI_wjv0vSDVlijMnZ9-Lkgqc--Bu80jwWL-CoTBjbpe1sae_INrRIGeFqgwK-ezXnEOgqpG2NO5P97SEgPcY4zY_QgnaTA6xv-rzef5kDF5s4pU5M8QdM-T8eUF020_v3zsmgR78rxXaglg0C5QuxYvT8R4THTJZuwa_f26Bo Aug 08 02:28:55.357558 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Id: None Aug 08 02:28:55.357558 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Name: None Aug 08 02:28:55.357558 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Identity-Status: Confirmed Aug 08 02:28:55.357558 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Is-Admin-Project: True Aug 08 02:28:55.357558 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Id: default Aug 08 02:28:55.357558 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Name: Default Aug 08 02:28:55.357558 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Id: ca29b2b9a59d4826bfd25f4dcfa9cffe Aug 08 02:28:55.357558 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Name: tempest-AttachVolumeNegativeTest-420450256 Aug 08 02:28:55.357558 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Role: reader,member Aug 08 02:28:55.357558 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Roles: reader,member Aug 08 02:28:55.361225 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/ca29b2b9a59d4826bfd25f4dcfa9cffe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/ca29b2b9a59d4826bfd25f4dcfa9cffe"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_ca29b2b9a59d4826bfd25f4dcfa9cffe", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/ca29b2b9a59d4826bfd25f4dcfa9cffe"}]}] Aug 08 02:28:55.361225 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant: tempest-AttachVolumeNegativeTest-420450256 Aug 08 02:28:55.361225 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Id: ca29b2b9a59d4826bfd25f4dcfa9cffe Aug 08 02:28:55.361225 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-420450256 Aug 08 02:28:55.361225 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User: tempest-AttachVolumeNegativeTest-420450256-project Aug 08 02:28:55.361225 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Id: default Aug 08 02:28:55.361225 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Name: Default Aug 08 02:28:55.361225 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Id: 578cd837d9fe442587f8c6c8528334c0 Aug 08 02:28:55.361225 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Name: tempest-AttachVolumeNegativeTest-420450256-project Aug 08 02:28:55.361225 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Aug 08 02:28:55.361225 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: {"volumeAttachment": {"volumeId": "bacd7028-1966-45e9-bc6e-a6156c1d3960"}} Aug 08 02:28:55.361225 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: method: > Aug 08 02:28:55.362058 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: exception: Invalid volume: volume bacd7028-1966-45e9-bc6e-a6156c1d3960 is already attached to instances: 95ae6b45-35d7-43a7-a021-91bb545cac4e Aug 08 02:28:55.362058 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: action_args: {'server_id': 'dda0c0b0-ed8e-4d43-9b26-abe1bca05a61', 'body': {'volumeAttachment': {'volumeId': 'bacd7028-1966-45e9-bc6e-a6156c1d3960'}}} Aug 08 02:28:55.362058 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:28:55.362058 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:28:55.362058 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Aug 08 02:28:55.362058 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Aug 08 02:28:55.362058 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 08 02:28:55.362058 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 08 02:28:55.362058 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Aug 08 02:28:55.362058 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 08 02:28:55.362058 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5066, in attach_volume Aug 08 02:28:55.362058 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Aug 08 02:28:55.362058 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4906, in _check_volume_already_attached Aug 08 02:28:55.362058 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Aug 08 02:28:55.362058 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume bacd7028-1966-45e9-bc6e-a6156c1d3960 is already attached to instances: 95ae6b45-35d7-43a7-a021-91bb545cac4e Aug 08 02:28:55.362058 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:28:55.362058 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:28:55.362058 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:28:55.362058 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:28:55.363419 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:28:55.363419 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:28:55.363419 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:28:55.363419 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:28:55.363419 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:28:55.363419 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:28:55.363419 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 08 02:28:55.363419 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 08 02:28:55.363419 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 08 02:28:55.363419 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 08 02:28:55.363419 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 08 02:28:55.363419 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 08 02:28:55.363419 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Aug 08 02:28:55.363419 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 08 02:28:55.363419 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume bacd7028-1966-45e9-bc6e-a6156c1d3960 is already attached to instances: 95ae6b45-35d7-43a7-a021-91bb545cac4e Aug 08 02:28:55.363419 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:28:55.363419 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:28:55.363419 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:28:55.363419 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:28:55.363419 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:28:55.364881 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:28:55.364881 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:28:55.364881 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:28:55.364881 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume bacd7028-1966-45e9-bc6e-a6156c1d3960 is already attached to instances: 95ae6b45-35d7-43a7-a021-91bb545cac4e Aug 08 02:28:55.364881 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:28:55.364881 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-403a6362-8ed6-4281-b198-3f2c662a70e7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Returning 400 to user: Invalid volume: volume bacd7028-1966-45e9-bc6e-a6156c1d3960 is already attached to instances: 95ae6b45-35d7-43a7-a021-91bb545cac4e {{(pid=118352) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:28:55.369138 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-403a6362-8ed6-4281-b198-3f2c662a70e7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "POST /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.147360 Aug 08 02:28:55.370193 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 221/443] 10.222.0.27 () {60 vars in 1323 bytes} [Mon Aug 8 02:28:55 2022] POST /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61/os-volume_attachments => generated 174 bytes in 158 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Aug 08 02:28:55.378968 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-cfc23ed3-f8c0-4750-ad24-d2d5504d8ebd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:55.379528 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-cfc23ed3-f8c0-4750-ad24-d2d5504d8ebd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: dda0c0b0-ed8e-4d43-9b26-abe1bca05a61] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:55.386373 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-cfc23ed3-f8c0-4750-ad24-d2d5504d8ebd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:55.386883 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-cfc23ed3-f8c0-4750-ad24-d2d5504d8ebd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:55.387602 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-cfc23ed3-f8c0-4750-ad24-d2d5504d8ebd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:55.388903 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-cfc23ed3-f8c0-4750-ad24-d2d5504d8ebd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:55.389625 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-cfc23ed3-f8c0-4750-ad24-d2d5504d8ebd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:55.390260 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-cfc23ed3-f8c0-4750-ad24-d2d5504d8ebd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:55.421817 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-cfc23ed3-f8c0-4750-ad24-d2d5504d8ebd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: dda0c0b0-ed8e-4d43-9b26-abe1bca05a61] Going to try to terminate instance {{(pid=118351) delete /opt/stack/nova/nova/compute/api.py:2666}} Aug 08 02:28:55.554167 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-2febcba7-e037-4fe0-b418-ed0520122a60 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:55.555199 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-2febcba7-e037-4fe0-b418-ed0520122a60 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:55.565261 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2febcba7-e037-4fe0-b418-ed0520122a60 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:55.565770 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2febcba7-e037-4fe0-b418-ed0520122a60 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:55.566478 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2febcba7-e037-4fe0-b418-ed0520122a60 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:55.567227 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2febcba7-e037-4fe0-b418-ed0520122a60 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:55.567675 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2febcba7-e037-4fe0-b418-ed0520122a60 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:55.568152 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2febcba7-e037-4fe0-b418-ed0520122a60 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:55.758881 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-cfc23ed3-f8c0-4750-ad24-d2d5504d8ebd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "DELETE /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61" status: 204 len: 0 microversion: 2.1 time: 0.383174 Aug 08 02:28:55.762616 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 223/444] 10.222.0.27 () {58 vars in 1239 bytes} [Mon Aug 8 02:28:55 2022] DELETE /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 08 02:28:55.768372 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-eb5e8ebf-8002-4ac2-b9f6-87046fd23554 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:55.771232 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-eb5e8ebf-8002-4ac2-b9f6-87046fd23554 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:55.786634 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-eb5e8ebf-8002-4ac2-b9f6-87046fd23554 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:55.787132 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-eb5e8ebf-8002-4ac2-b9f6-87046fd23554 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:55.787582 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-eb5e8ebf-8002-4ac2-b9f6-87046fd23554 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:55.788300 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-eb5e8ebf-8002-4ac2-b9f6-87046fd23554 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:55.791725 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-eb5e8ebf-8002-4ac2-b9f6-87046fd23554 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:55.792218 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-eb5e8ebf-8002-4ac2-b9f6-87046fd23554 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:55.799426 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-2febcba7-e037-4fe0-b418-ed0520122a60 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:55.816441 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-2febcba7-e037-4fe0-b418-ed0520122a60 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1680 microversion: 2.1 time: 0.267414 Aug 08 02:28:55.817127 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 222/445] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:28:55 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1680 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:28:55.829248 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-6233497f-6e16-4c28-a084-7a6bb31cb4d4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:55.835636 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-6233497f-6e16-4c28-a084-7a6bb31cb4d4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: dda0c0b0-ed8e-4d43-9b26-abe1bca05a61] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:55.841471 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6233497f-6e16-4c28-a084-7a6bb31cb4d4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:55.842087 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6233497f-6e16-4c28-a084-7a6bb31cb4d4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:55.842689 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6233497f-6e16-4c28-a084-7a6bb31cb4d4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:55.843682 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6233497f-6e16-4c28-a084-7a6bb31cb4d4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:55.844342 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6233497f-6e16-4c28-a084-7a6bb31cb4d4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:55.844901 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6233497f-6e16-4c28-a084-7a6bb31cb4d4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:56.081856 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-eb5e8ebf-8002-4ac2-b9f6-87046fd23554 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:56.090469 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-eb5e8ebf-8002-4ac2-b9f6-87046fd23554 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1736 microversion: 2.1 time: 0.328960 Aug 08 02:28:56.093030 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 224/446] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:28:55 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1736 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:28:56.122351 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-6233497f-6e16-4c28-a084-7a6bb31cb4d4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:56.135436 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-6233497f-6e16-4c28-a084-7a6bb31cb4d4 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61" status: 200 len: 1683 microversion: 2.1 time: 0.316193 Aug 08 02:28:56.136576 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 223/447] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:28:55 2022] GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 => generated 1683 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:28:56.383002 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-4ff840ae-c5ac-436e-87a3-9081d7bf5541 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Action: 'create', calling method: .version_select at 0x7efc7427bee0>, body: {"keypair": {"name": "tempest-TestShelveInstance-556747333", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPd9cA0JlZ2LLZAfpLpJJc6cLnPTH7ncualiW0OZyGBV0LL0HcHj5JtDh3l2cMNQXUi0ZpmbbJtEb6+pxrd+bNTu/BOCG3sBjMdcqHFDIqLTowhitQzBvMLSnfQEmNNJIA=="}} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:28:56.400268 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.quota [None req-4ff840ae-c5ac-436e-87a3-9081d7bf5541 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Getting quotas for project fa4edbfca7b749ff9be3268fa6b81264. Resources: {'key_pairs'} {{(pid=118351) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 08 02:28:56.404812 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.quota [None req-4ff840ae-c5ac-436e-87a3-9081d7bf5541 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Getting quotas for user 8be4cb4f1c594f42825eb40985a29cac and project fa4edbfca7b749ff9be3268fa6b81264. Resources: {'key_pairs'} {{(pid=118351) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 08 02:28:56.495187 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-4ff840ae-c5ac-436e-87a3-9081d7bf5541 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "POST /compute/v2.1/os-keypairs" status: 200 len: 396 microversion: 2.1 time: 0.331426 Aug 08 02:28:56.495794 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 225/448] 10.222.0.27 () {60 vars in 1159 bytes} [Mon Aug 8 02:28:56 2022] POST /compute/v2.1/os-keypairs => generated 396 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:28:56.842281 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-a79d3787-8cde-4faf-a3e7-fc8acc229274 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:56.843344 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-a79d3787-8cde-4faf-a3e7-fc8acc229274 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:56.851266 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a79d3787-8cde-4faf-a3e7-fc8acc229274 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:56.851912 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a79d3787-8cde-4faf-a3e7-fc8acc229274 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:56.852549 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a79d3787-8cde-4faf-a3e7-fc8acc229274 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:56.853363 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a79d3787-8cde-4faf-a3e7-fc8acc229274 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:56.854049 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a79d3787-8cde-4faf-a3e7-fc8acc229274 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:56.854573 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a79d3787-8cde-4faf-a3e7-fc8acc229274 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:57.042587 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-a79d3787-8cde-4faf-a3e7-fc8acc229274 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:57.051523 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-a79d3787-8cde-4faf-a3e7-fc8acc229274 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1680 microversion: 2.1 time: 0.213046 Aug 08 02:28:57.052106 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 224/449] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:28:56 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1680 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:28:57.114107 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-abbd0fae-db97-40ce-9e9b-9d66ca523356 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:57.114722 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-abbd0fae-db97-40ce-9e9b-9d66ca523356 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:57.121658 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-abbd0fae-db97-40ce-9e9b-9d66ca523356 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:57.122148 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-abbd0fae-db97-40ce-9e9b-9d66ca523356 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:57.122586 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-abbd0fae-db97-40ce-9e9b-9d66ca523356 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:57.123285 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-abbd0fae-db97-40ce-9e9b-9d66ca523356 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:57.123739 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-abbd0fae-db97-40ce-9e9b-9d66ca523356 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:57.124817 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-abbd0fae-db97-40ce-9e9b-9d66ca523356 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:57.162562 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-66f6c634-1178-46b6-aed3-b3ed87a61813 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:57.163684 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-66f6c634-1178-46b6-aed3-b3ed87a61813 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: dda0c0b0-ed8e-4d43-9b26-abe1bca05a61] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:57.170292 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-66f6c634-1178-46b6-aed3-b3ed87a61813 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:57.174898 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-66f6c634-1178-46b6-aed3-b3ed87a61813 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:57.174898 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-66f6c634-1178-46b6-aed3-b3ed87a61813 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:57.174898 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-66f6c634-1178-46b6-aed3-b3ed87a61813 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:57.174898 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-66f6c634-1178-46b6-aed3-b3ed87a61813 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:57.175300 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-66f6c634-1178-46b6-aed3-b3ed87a61813 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:57.366231 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-abbd0fae-db97-40ce-9e9b-9d66ca523356 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:57.376181 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-abbd0fae-db97-40ce-9e9b-9d66ca523356 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1736 microversion: 2.1 time: 0.265977 Aug 08 02:28:57.377390 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 226/450] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:28:57 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1736 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:28:57.385933 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-66f6c634-1178-46b6-aed3-b3ed87a61813 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:57.394295 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-66f6c634-1178-46b6-aed3-b3ed87a61813 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61" status: 200 len: 1683 microversion: 2.1 time: 0.239193 Aug 08 02:28:57.394860 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 225/451] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:28:57 2022] GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 => generated 1683 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:28:58.074656 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-8fea6a21-3c5b-408d-b5b5-06014b617f85 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:58.076023 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-8fea6a21-3c5b-408d-b5b5-06014b617f85 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:58.084030 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8fea6a21-3c5b-408d-b5b5-06014b617f85 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:58.084533 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8fea6a21-3c5b-408d-b5b5-06014b617f85 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:58.085015 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8fea6a21-3c5b-408d-b5b5-06014b617f85 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:58.085728 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8fea6a21-3c5b-408d-b5b5-06014b617f85 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:58.086406 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8fea6a21-3c5b-408d-b5b5-06014b617f85 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:58.086942 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8fea6a21-3c5b-408d-b5b5-06014b617f85 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:58.257819 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-8fea6a21-3c5b-408d-b5b5-06014b617f85 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:58.270657 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-8fea6a21-3c5b-408d-b5b5-06014b617f85 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1680 microversion: 2.1 time: 0.199890 Aug 08 02:28:58.271078 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 227/452] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:28:58 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1680 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:28:58.398431 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-347de02e-63d4-4906-8f84-01f6779a4db2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:58.399900 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-347de02e-63d4-4906-8f84-01f6779a4db2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:58.407084 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-347de02e-63d4-4906-8f84-01f6779a4db2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:58.407576 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-347de02e-63d4-4906-8f84-01f6779a4db2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:58.408006 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-347de02e-63d4-4906-8f84-01f6779a4db2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:58.408966 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-347de02e-63d4-4906-8f84-01f6779a4db2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:58.409635 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-347de02e-63d4-4906-8f84-01f6779a4db2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:58.410266 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-347de02e-63d4-4906-8f84-01f6779a4db2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:58.416490 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-1084d531-05da-42b9-81d5-ef47a40853cc tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:58.419452 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-1084d531-05da-42b9-81d5-ef47a40853cc tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: dda0c0b0-ed8e-4d43-9b26-abe1bca05a61] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:58.430158 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1084d531-05da-42b9-81d5-ef47a40853cc tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:58.431564 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1084d531-05da-42b9-81d5-ef47a40853cc tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:58.432239 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1084d531-05da-42b9-81d5-ef47a40853cc tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:58.433209 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1084d531-05da-42b9-81d5-ef47a40853cc tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:58.434553 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1084d531-05da-42b9-81d5-ef47a40853cc tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:58.435175 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1084d531-05da-42b9-81d5-ef47a40853cc tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:58.630503 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-347de02e-63d4-4906-8f84-01f6779a4db2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:58.637019 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-1084d531-05da-42b9-81d5-ef47a40853cc tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:58.639154 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-347de02e-63d4-4906-8f84-01f6779a4db2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1736 microversion: 2.1 time: 0.244948 Aug 08 02:28:58.640009 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 226/453] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:28:58 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1736 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:28:58.644769 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-1084d531-05da-42b9-81d5-ef47a40853cc tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61" status: 200 len: 1683 microversion: 2.1 time: 0.233061 Aug 08 02:28:58.645350 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 228/454] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:28:58 2022] GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 => generated 1683 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:28:59.291366 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-6f182f1f-7ab1-4bc0-ab1d-71d45b02d807 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:59.291978 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-6f182f1f-7ab1-4bc0-ab1d-71d45b02d807 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:59.299436 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6f182f1f-7ab1-4bc0-ab1d-71d45b02d807 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:59.300052 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6f182f1f-7ab1-4bc0-ab1d-71d45b02d807 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:59.300350 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6f182f1f-7ab1-4bc0-ab1d-71d45b02d807 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:59.301084 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6f182f1f-7ab1-4bc0-ab1d-71d45b02d807 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:59.301572 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6f182f1f-7ab1-4bc0-ab1d-71d45b02d807 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:59.301996 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6f182f1f-7ab1-4bc0-ab1d-71d45b02d807 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:59.451587 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-6f182f1f-7ab1-4bc0-ab1d-71d45b02d807 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:59.461497 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-6f182f1f-7ab1-4bc0-ab1d-71d45b02d807 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1680 microversion: 2.1 time: 0.173946 Aug 08 02:28:59.462110 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 227/455] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:28:59 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1680 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:28:59.660795 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-3944e799-e12a-43bb-bc28-4c9a152c98d9 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:59.661484 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-3944e799-e12a-43bb-bc28-4c9a152c98d9 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:59.666080 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-82041405-4223-4b94-bbad-c6c9525894c1 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:28:59.666699 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-82041405-4223-4b94-bbad-c6c9525894c1 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: dda0c0b0-ed8e-4d43-9b26-abe1bca05a61] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:28:59.668188 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3944e799-e12a-43bb-bc28-4c9a152c98d9 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:59.668912 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3944e799-e12a-43bb-bc28-4c9a152c98d9 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:59.669350 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3944e799-e12a-43bb-bc28-4c9a152c98d9 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:59.670050 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3944e799-e12a-43bb-bc28-4c9a152c98d9 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:59.670514 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3944e799-e12a-43bb-bc28-4c9a152c98d9 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:59.671094 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3944e799-e12a-43bb-bc28-4c9a152c98d9 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:59.673534 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-82041405-4223-4b94-bbad-c6c9525894c1 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:59.674150 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-82041405-4223-4b94-bbad-c6c9525894c1 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:59.674594 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-82041405-4223-4b94-bbad-c6c9525894c1 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:59.675344 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-82041405-4223-4b94-bbad-c6c9525894c1 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:28:59.675810 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-82041405-4223-4b94-bbad-c6c9525894c1 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:28:59.676244 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-82041405-4223-4b94-bbad-c6c9525894c1 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:28:59.892083 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-3944e799-e12a-43bb-bc28-4c9a152c98d9 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:59.896651 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-82041405-4223-4b94-bbad-c6c9525894c1 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:28:59.905898 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-82041405-4223-4b94-bbad-c6c9525894c1 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61" status: 200 len: 1683 microversion: 2.1 time: 0.243344 Aug 08 02:28:59.920652 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 228/456] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:28:59 2022] GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 => generated 1683 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:28:59.927801 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-3944e799-e12a-43bb-bc28-4c9a152c98d9 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1736 microversion: 2.1 time: 0.271119 Aug 08 02:28:59.933038 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 229/457] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:28:59 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1736 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:00.484301 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-69a72b6d-430b-4e8a-9320-006ac3fabe4b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:00.485030 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-69a72b6d-430b-4e8a-9320-006ac3fabe4b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:00.491663 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-69a72b6d-430b-4e8a-9320-006ac3fabe4b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:00.496175 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-69a72b6d-430b-4e8a-9320-006ac3fabe4b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:00.496175 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-69a72b6d-430b-4e8a-9320-006ac3fabe4b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:00.496175 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-69a72b6d-430b-4e8a-9320-006ac3fabe4b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:00.496175 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-69a72b6d-430b-4e8a-9320-006ac3fabe4b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:00.496506 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-69a72b6d-430b-4e8a-9320-006ac3fabe4b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:00.647200 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-69a72b6d-430b-4e8a-9320-006ac3fabe4b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:00.657975 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-69a72b6d-430b-4e8a-9320-006ac3fabe4b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1680 microversion: 2.1 time: 0.177780 Aug 08 02:29:00.659307 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 229/458] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:00 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1680 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:00.873697 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [req-b79e7f7d-3150-48ee-a6c8-70ee92fbd25f req-af315dd1-8791-43be-b226-0ea918808555 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dda0c0b0-ed8e-4d43-9b26-abe1bca05a61", "name": "network-vif-deleted", "tag": "c32805ad-aa79-4a4e-bcb8-fdaa7a625107"}]} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:29:00.888978 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-b79e7f7d-3150-48ee-a6c8-70ee92fbd25f req-af315dd1-8791-43be-b226-0ea918808555 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:00.891325 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-b79e7f7d-3150-48ee-a6c8-70ee92fbd25f req-af315dd1-8791-43be-b226-0ea918808555 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:00.896796 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-b79e7f7d-3150-48ee-a6c8-70ee92fbd25f req-af315dd1-8791-43be-b226-0ea918808555 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:00.909221 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.compute.server_external_events [req-b79e7f7d-3150-48ee-a6c8-70ee92fbd25f req-af315dd1-8791-43be-b226-0ea918808555 service nova] Creating event network-vif-deleted:c32805ad-aa79-4a4e-bcb8-fdaa7a625107 for instance dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 on hv-cinder-74014 Aug 08 02:29:00.915047 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [req-b79e7f7d-3150-48ee-a6c8-70ee92fbd25f req-af315dd1-8791-43be-b226-0ea918808555 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044981 Aug 08 02:29:00.915809 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 230/459] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:29:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:29:00.927521 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-ea88f355-2c58-4fba-be1d-5c69dffa5fa6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:00.929888 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-ea88f355-2c58-4fba-be1d-5c69dffa5fa6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: dda0c0b0-ed8e-4d43-9b26-abe1bca05a61] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:00.934660 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ea88f355-2c58-4fba-be1d-5c69dffa5fa6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:00.935231 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ea88f355-2c58-4fba-be1d-5c69dffa5fa6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:00.935705 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ea88f355-2c58-4fba-be1d-5c69dffa5fa6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:00.936761 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ea88f355-2c58-4fba-be1d-5c69dffa5fa6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:00.937422 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ea88f355-2c58-4fba-be1d-5c69dffa5fa6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:00.938012 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ea88f355-2c58-4fba-be1d-5c69dffa5fa6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:00.960932 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-5006a6d9-7a6b-48fd-b335-5bb0d2ffe26b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:00.963397 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-5006a6d9-7a6b-48fd-b335-5bb0d2ffe26b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:00.972621 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5006a6d9-7a6b-48fd-b335-5bb0d2ffe26b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:00.973289 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5006a6d9-7a6b-48fd-b335-5bb0d2ffe26b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:00.973991 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5006a6d9-7a6b-48fd-b335-5bb0d2ffe26b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:00.974875 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5006a6d9-7a6b-48fd-b335-5bb0d2ffe26b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:00.975558 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5006a6d9-7a6b-48fd-b335-5bb0d2ffe26b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:00.976200 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5006a6d9-7a6b-48fd-b335-5bb0d2ffe26b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:01.023510 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-ea88f355-2c58-4fba-be1d-5c69dffa5fa6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:01.030765 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-ea88f355-2c58-4fba-be1d-5c69dffa5fa6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61" status: 200 len: 1615 microversion: 2.1 time: 0.106702 Aug 08 02:29:01.031941 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 230/460] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:00 2022] GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 => generated 1615 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:01.184457 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-5006a6d9-7a6b-48fd-b335-5bb0d2ffe26b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:01.204372 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-5006a6d9-7a6b-48fd-b335-5bb0d2ffe26b tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1736 microversion: 2.1 time: 0.257114 Aug 08 02:29:01.205608 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 231/461] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:00 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1736 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:01.680886 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-079cc5ce-28f5-4b79-ba62-c653b155f3fd tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:01.685656 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-079cc5ce-28f5-4b79-ba62-c653b155f3fd tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:01.694800 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-079cc5ce-28f5-4b79-ba62-c653b155f3fd tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:01.695656 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-079cc5ce-28f5-4b79-ba62-c653b155f3fd tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:01.696311 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-079cc5ce-28f5-4b79-ba62-c653b155f3fd tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:01.698321 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-079cc5ce-28f5-4b79-ba62-c653b155f3fd tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:01.699601 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-079cc5ce-28f5-4b79-ba62-c653b155f3fd tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:01.700303 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-079cc5ce-28f5-4b79-ba62-c653b155f3fd tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:01.970671 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-079cc5ce-28f5-4b79-ba62-c653b155f3fd tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:01.980500 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-079cc5ce-28f5-4b79-ba62-c653b155f3fd tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1680 microversion: 2.1 time: 0.303481 Aug 08 02:29:01.981180 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 231/462] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:01 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1680 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:02.053695 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-2da22014-561a-4eff-880c-e7388cae671c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:02.054145 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-2da22014-561a-4eff-880c-e7388cae671c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: dda0c0b0-ed8e-4d43-9b26-abe1bca05a61] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:02.071229 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2da22014-561a-4eff-880c-e7388cae671c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:02.071763 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2da22014-561a-4eff-880c-e7388cae671c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:02.072215 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2da22014-561a-4eff-880c-e7388cae671c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:02.073018 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2da22014-561a-4eff-880c-e7388cae671c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:02.073534 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2da22014-561a-4eff-880c-e7388cae671c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:02.074011 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2da22014-561a-4eff-880c-e7388cae671c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:02.135589 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2da22014-561a-4eff-880c-e7388cae671c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:02.136374 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2da22014-561a-4eff-880c-e7388cae671c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:02.137018 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2da22014-561a-4eff-880c-e7388cae671c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:02.137662 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.objects.instance [None req-2da22014-561a-4eff-880c-e7388cae671c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lazy-loading 'fault' on Instance uuid dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 {{(pid=118351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 08 02:29:02.204240 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-2da22014-561a-4eff-880c-e7388cae671c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:02.214909 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-2da22014-561a-4eff-880c-e7388cae671c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61" status: 200 len: 1331 microversion: 2.1 time: 0.165499 Aug 08 02:29:02.229360 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 232/463] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:02 2022] GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 => generated 1331 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:02.237546 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-119cc932-79e0-4660-884c-8555181153be tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:02.238517 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-119cc932-79e0-4660-884c-8555181153be tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:02.246645 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-119cc932-79e0-4660-884c-8555181153be tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:02.247531 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-119cc932-79e0-4660-884c-8555181153be tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:02.248191 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-119cc932-79e0-4660-884c-8555181153be tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:02.249709 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-119cc932-79e0-4660-884c-8555181153be tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:02.250390 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-119cc932-79e0-4660-884c-8555181153be tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:02.251490 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-119cc932-79e0-4660-884c-8555181153be tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:02.434547 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-119cc932-79e0-4660-884c-8555181153be tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:02.459616 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-119cc932-79e0-4660-884c-8555181153be tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1736 microversion: 2.1 time: 0.227906 Aug 08 02:29:02.463884 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 232/464] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:02 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1736 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:03.005259 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-05de4861-4274-4764-97dd-b419b09c1a86 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:03.006220 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-05de4861-4274-4764-97dd-b419b09c1a86 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:03.013268 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-05de4861-4274-4764-97dd-b419b09c1a86 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:03.014002 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-05de4861-4274-4764-97dd-b419b09c1a86 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:03.014779 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-05de4861-4274-4764-97dd-b419b09c1a86 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:03.016141 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-05de4861-4274-4764-97dd-b419b09c1a86 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:03.017057 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-05de4861-4274-4764-97dd-b419b09c1a86 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:03.017547 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-05de4861-4274-4764-97dd-b419b09c1a86 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:03.152222 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-05de4861-4274-4764-97dd-b419b09c1a86 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:03.161205 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-05de4861-4274-4764-97dd-b419b09c1a86 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1680 microversion: 2.1 time: 0.160409 Aug 08 02:29:03.161776 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 233/465] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:02 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1680 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:03.242620 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-bf037a88-3fc5-499e-b14c-3d1aade82d73 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:03.243266 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-bf037a88-3fc5-499e-b14c-3d1aade82d73 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: dda0c0b0-ed8e-4d43-9b26-abe1bca05a61] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:03.250101 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-bf037a88-3fc5-499e-b14c-3d1aade82d73 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:03.250681 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-bf037a88-3fc5-499e-b14c-3d1aade82d73 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:03.251122 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-bf037a88-3fc5-499e-b14c-3d1aade82d73 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:03.251841 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-bf037a88-3fc5-499e-b14c-3d1aade82d73 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:03.252298 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-bf037a88-3fc5-499e-b14c-3d1aade82d73 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:03.252745 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-bf037a88-3fc5-499e-b14c-3d1aade82d73 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:03.267673 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.wsgi [None req-bf037a88-3fc5-499e-b14c-3d1aade82d73 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] HTTP exception thrown: Instance dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 could not be found. Aug 08 02:29:03.269257 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-bf037a88-3fc5-499e-b14c-3d1aade82d73 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Request method failure captured: Aug 08 02:29:03.269257 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: request: GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 HTTP/1.1 Aug 08 02:29:03.269257 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept: application/json Aug 08 02:29:03.269257 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept-Encoding: identity Aug 08 02:29:03.269257 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Connection: close Aug 08 02:29:03.269257 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Length: 0 Aug 08 02:29:03.269257 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Type: application/json Aug 08 02:29:03.269257 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Host: 10.222.0.27 Aug 08 02:29:03.269257 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Openstack-System-Scope: None Aug 08 02:29:03.269257 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: User-Agent: python-urllib3/1.26.11 Aug 08 02:29:03.269257 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Auth-Token: gAAAAABi8HQUs7QI_wjv0vSDVlijMnZ9-Lkgqc--Bu80jwWL-CoTBjbpe1sae_INrRIGeFqgwK-ezXnEOgqpG2NO5P97SEgPcY4zY_QgnaTA6xv-rzef5kDF5s4pU5M8QdM-T8eUF020_v3zsmgR78rxXaglg0C5QuxYvT8R4THTJZuwa_f26Bo Aug 08 02:29:03.269257 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Id: None Aug 08 02:29:03.269257 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Name: None Aug 08 02:29:03.269257 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Identity-Status: Confirmed Aug 08 02:29:03.269257 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Is-Admin-Project: True Aug 08 02:29:03.269257 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Id: default Aug 08 02:29:03.269257 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Name: Default Aug 08 02:29:03.269257 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Id: ca29b2b9a59d4826bfd25f4dcfa9cffe Aug 08 02:29:03.269257 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Name: tempest-AttachVolumeNegativeTest-420450256 Aug 08 02:29:03.269257 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Role: reader,member Aug 08 02:29:03.269257 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Roles: reader,member Aug 08 02:29:03.270798 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/ca29b2b9a59d4826bfd25f4dcfa9cffe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/ca29b2b9a59d4826bfd25f4dcfa9cffe"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_ca29b2b9a59d4826bfd25f4dcfa9cffe", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/ca29b2b9a59d4826bfd25f4dcfa9cffe"}]}] Aug 08 02:29:03.270798 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant: tempest-AttachVolumeNegativeTest-420450256 Aug 08 02:29:03.270798 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Id: ca29b2b9a59d4826bfd25f4dcfa9cffe Aug 08 02:29:03.270798 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-420450256 Aug 08 02:29:03.270798 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User: tempest-AttachVolumeNegativeTest-420450256-project Aug 08 02:29:03.270798 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Id: default Aug 08 02:29:03.270798 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Name: Default Aug 08 02:29:03.270798 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Id: 578cd837d9fe442587f8c6c8528334c0 Aug 08 02:29:03.270798 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Name: tempest-AttachVolumeNegativeTest-420450256-project Aug 08 02:29:03.270798 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: method: > Aug 08 02:29:03.270798 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: exception: Instance dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 could not be found. Aug 08 02:29:03.270798 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: action_args: {'id': 'dda0c0b0-ed8e-4d43-9b26-abe1bca05a61'} Aug 08 02:29:03.271776 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:29:03.271776 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:29:03.271776 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 08 02:29:03.271776 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 08 02:29:03.271776 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 08 02:29:03.271776 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 08 02:29:03.271776 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 08 02:29:03.271776 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 08 02:29:03.271776 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 08 02:29:03.271776 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 08 02:29:03.271776 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 could not be found. Aug 08 02:29:03.271776 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:29:03.271776 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:29:03.271776 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:29:03.271776 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:29:03.271776 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:29:03.271776 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:29:03.271776 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:29:03.271776 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:29:03.271776 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:29:03.273312 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:29:03.273312 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 08 02:29:03.273312 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 08 02:29:03.273312 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 08 02:29:03.273312 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 08 02:29:03.273312 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 08 02:29:03.273312 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:29:03.273312 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 could not be found. Aug 08 02:29:03.273312 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:29:03.273312 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:29:03.273312 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:29:03.273312 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:29:03.273312 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:29:03.273312 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:29:03.273312 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:29:03.273312 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:29:03.273312 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 could not be found. Aug 08 02:29:03.273312 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:29:03.274763 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-bf037a88-3fc5-499e-b14c-3d1aade82d73 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Returning 404 to user: Instance dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 could not be found. {{(pid=118352) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:29:03.274763 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-bf037a88-3fc5-499e-b14c-3d1aade82d73 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61" status: 404 len: 111 microversion: 2.1 time: 0.032216 Aug 08 02:29:03.274763 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 233/466] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:03 2022] GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 08 02:29:03.279681 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-1552e619-4fb0-4a6b-b266-0f16475c67b6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '.version_select at 0x7efc761bddc0>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:03.392716 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-1552e619-4fb0-4a6b-b266-0f16475c67b6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-458871973" status: 202 len: 0 microversion: 2.1 time: 0.116808 Aug 08 02:29:03.393299 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 234/467] 10.222.0.27 () {58 vars in 1218 bytes} [Mon Aug 8 02:29:03 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-458871973 => generated 0 bytes in 118 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 08 02:29:03.486895 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-34b35d17-7c4c-483c-9a0d-3a7439522320 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:03.487643 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-34b35d17-7c4c-483c-9a0d-3a7439522320 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:03.494382 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-34b35d17-7c4c-483c-9a0d-3a7439522320 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:03.495189 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-34b35d17-7c4c-483c-9a0d-3a7439522320 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:03.495902 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-34b35d17-7c4c-483c-9a0d-3a7439522320 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:03.497260 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-34b35d17-7c4c-483c-9a0d-3a7439522320 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:03.497833 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-34b35d17-7c4c-483c-9a0d-3a7439522320 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:03.498532 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-34b35d17-7c4c-483c-9a0d-3a7439522320 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:03.635541 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-34b35d17-7c4c-483c-9a0d-3a7439522320 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:03.644255 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-34b35d17-7c4c-483c-9a0d-3a7439522320 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1736 microversion: 2.1 time: 0.161697 Aug 08 02:29:03.644850 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 234/468] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:03 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1736 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:04.185916 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-93c5df42-4ded-4878-9ddb-463c5d8219a6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:04.187132 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-93c5df42-4ded-4878-9ddb-463c5d8219a6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:04.194424 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-93c5df42-4ded-4878-9ddb-463c5d8219a6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:04.194947 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-93c5df42-4ded-4878-9ddb-463c5d8219a6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:04.195399 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-93c5df42-4ded-4878-9ddb-463c5d8219a6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:04.196087 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-93c5df42-4ded-4878-9ddb-463c5d8219a6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:04.196539 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-93c5df42-4ded-4878-9ddb-463c5d8219a6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:04.196994 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-93c5df42-4ded-4878-9ddb-463c5d8219a6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:04.348620 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-526800fe-c77f-4820-90fe-80913b949166 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:04.349206 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-526800fe-c77f-4820-90fe-80913b949166 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: 95ae6b45-35d7-43a7-a021-91bb545cac4e] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:04.352404 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-93c5df42-4ded-4878-9ddb-463c5d8219a6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:04.356029 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-526800fe-c77f-4820-90fe-80913b949166 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:04.356602 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-526800fe-c77f-4820-90fe-80913b949166 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:04.356980 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:29:04.357192 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-526800fe-c77f-4820-90fe-80913b949166 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:04.358836 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-526800fe-c77f-4820-90fe-80913b949166 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:04.359814 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-526800fe-c77f-4820-90fe-80913b949166 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:04.360496 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-526800fe-c77f-4820-90fe-80913b949166 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:04.365397 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-93c5df42-4ded-4878-9ddb-463c5d8219a6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1680 microversion: 2.1 time: 0.186591 Aug 08 02:29:04.365982 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 235/469] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:04 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1680 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:04.499268 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.objects.instance [None req-526800fe-c77f-4820-90fe-80913b949166 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lazy-loading 'services' on Instance uuid 95ae6b45-35d7-43a7-a021-91bb545cac4e {{(pid=118352) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 08 02:29:04.665762 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-cc3806c5-aa0e-4e56-8733-3e62a314513c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:04.666396 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-cc3806c5-aa0e-4e56-8733-3e62a314513c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:04.674562 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-cc3806c5-aa0e-4e56-8733-3e62a314513c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:04.675053 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-cc3806c5-aa0e-4e56-8733-3e62a314513c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:04.675492 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-cc3806c5-aa0e-4e56-8733-3e62a314513c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:04.676199 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-cc3806c5-aa0e-4e56-8733-3e62a314513c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:04.676653 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-cc3806c5-aa0e-4e56-8733-3e62a314513c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:04.677108 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-cc3806c5-aa0e-4e56-8733-3e62a314513c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:04.820680 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-cc3806c5-aa0e-4e56-8733-3e62a314513c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:04.829974 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-cc3806c5-aa0e-4e56-8733-3e62a314513c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1736 microversion: 2.1 time: 0.167953 Aug 08 02:29:04.830543 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 236/470] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:04 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1736 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:29:04.858549 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-526800fe-c77f-4820-90fe-80913b949166 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "DELETE /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e/os-volume_attachments/bacd7028-1966-45e9-bc6e-a6156c1d3960" status: 202 len: 0 microversion: 2.1 time: 0.513671 Aug 08 02:29:04.859146 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 235/471] 10.222.0.27 () {58 vars in 1416 bytes} [Mon Aug 8 02:29:04 2022] DELETE /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e/os-volume_attachments/bacd7028-1966-45e9-bc6e-a6156c1d3960 => generated 0 bytes in 515 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Aug 08 02:29:04.869575 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-270c1c5a-231a-4c08-8972-70339b15efb9 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:04.871779 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-270c1c5a-231a-4c08-8972-70339b15efb9 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: 95ae6b45-35d7-43a7-a021-91bb545cac4e] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:04.877892 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-270c1c5a-231a-4c08-8972-70339b15efb9 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:04.878378 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-270c1c5a-231a-4c08-8972-70339b15efb9 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:04.878812 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-270c1c5a-231a-4c08-8972-70339b15efb9 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:04.879510 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-270c1c5a-231a-4c08-8972-70339b15efb9 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:04.880036 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-270c1c5a-231a-4c08-8972-70339b15efb9 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:04.880458 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-270c1c5a-231a-4c08-8972-70339b15efb9 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:04.910432 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-270c1c5a-231a-4c08-8972-70339b15efb9 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046749 Aug 08 02:29:04.910991 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 237/472] 10.222.0.27 () {58 vars in 1302 bytes} [Mon Aug 8 02:29:04 2022] GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:29:05.390322 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-e184f700-6e11-45ad-ad4b-8c812fa04b99 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:05.391079 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-e184f700-6e11-45ad-ad4b-8c812fa04b99 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:05.398751 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e184f700-6e11-45ad-ad4b-8c812fa04b99 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:05.399250 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e184f700-6e11-45ad-ad4b-8c812fa04b99 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:05.399711 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e184f700-6e11-45ad-ad4b-8c812fa04b99 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:05.400422 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e184f700-6e11-45ad-ad4b-8c812fa04b99 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:05.400910 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e184f700-6e11-45ad-ad4b-8c812fa04b99 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:05.401352 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e184f700-6e11-45ad-ad4b-8c812fa04b99 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:05.536832 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-e184f700-6e11-45ad-ad4b-8c812fa04b99 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:05.544380 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-e184f700-6e11-45ad-ad4b-8c812fa04b99 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1680 microversion: 2.1 time: 0.159876 Aug 08 02:29:05.544971 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 236/473] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:05 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1680 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:05.851463 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-726d5682-6445-4280-b54e-d5557569a433 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:05.852141 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-726d5682-6445-4280-b54e-d5557569a433 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:05.859933 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-726d5682-6445-4280-b54e-d5557569a433 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:05.860431 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-726d5682-6445-4280-b54e-d5557569a433 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:05.860896 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-726d5682-6445-4280-b54e-d5557569a433 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:05.861618 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-726d5682-6445-4280-b54e-d5557569a433 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:05.862076 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-726d5682-6445-4280-b54e-d5557569a433 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:05.862526 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-726d5682-6445-4280-b54e-d5557569a433 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:05.923946 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-08c39af0-a0dc-4f36-8791-acc001d2ed25 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:05.926140 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-08c39af0-a0dc-4f36-8791-acc001d2ed25 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: 95ae6b45-35d7-43a7-a021-91bb545cac4e] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:05.932028 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-08c39af0-a0dc-4f36-8791-acc001d2ed25 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:05.932507 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-08c39af0-a0dc-4f36-8791-acc001d2ed25 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:05.932981 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-08c39af0-a0dc-4f36-8791-acc001d2ed25 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:05.933662 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-08c39af0-a0dc-4f36-8791-acc001d2ed25 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:05.934135 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-08c39af0-a0dc-4f36-8791-acc001d2ed25 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:05.934556 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-08c39af0-a0dc-4f36-8791-acc001d2ed25 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:05.954245 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:29:05.970347 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-08c39af0-a0dc-4f36-8791-acc001d2ed25 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051256 Aug 08 02:29:05.970902 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 237/474] 10.222.0.27 () {58 vars in 1302 bytes} [Mon Aug 8 02:29:05 2022] GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 08 02:29:05.996827 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-726d5682-6445-4280-b54e-d5557569a433 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:06.005140 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-726d5682-6445-4280-b54e-d5557569a433 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1736 microversion: 2.1 time: 0.157636 Aug 08 02:29:06.005704 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 238/475] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:05 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1736 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:06.568008 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-3e753ca0-31d7-402d-bd4d-26c660f2d10a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:06.568622 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-3e753ca0-31d7-402d-bd4d-26c660f2d10a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:06.575350 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3e753ca0-31d7-402d-bd4d-26c660f2d10a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:06.575830 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3e753ca0-31d7-402d-bd4d-26c660f2d10a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:06.576264 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3e753ca0-31d7-402d-bd4d-26c660f2d10a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:06.583709 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3e753ca0-31d7-402d-bd4d-26c660f2d10a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:06.583709 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3e753ca0-31d7-402d-bd4d-26c660f2d10a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:06.583709 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3e753ca0-31d7-402d-bd4d-26c660f2d10a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:06.714001 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-3e753ca0-31d7-402d-bd4d-26c660f2d10a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:06.722358 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-3e753ca0-31d7-402d-bd4d-26c660f2d10a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1680 microversion: 2.1 time: 0.158787 Aug 08 02:29:06.722935 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 238/476] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:06 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1680 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:07.040056 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-36a163fb-6d02-42e6-b9e3-7acea0d7ff80 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:07.040056 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-36a163fb-6d02-42e6-b9e3-7acea0d7ff80 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: 95ae6b45-35d7-43a7-a021-91bb545cac4e] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:07.040056 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell0:SELECT=4 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:29:07.040056 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-36a163fb-6d02-42e6-b9e3-7acea0d7ff80 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:07.040056 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-36a163fb-6d02-42e6-b9e3-7acea0d7ff80 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:07.040806 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-36a163fb-6d02-42e6-b9e3-7acea0d7ff80 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:07.040806 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-36a163fb-6d02-42e6-b9e3-7acea0d7ff80 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:07.040806 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-36a163fb-6d02-42e6-b9e3-7acea0d7ff80 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:07.040806 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-36a163fb-6d02-42e6-b9e3-7acea0d7ff80 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:07.041121 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-36a163fb-6d02-42e6-b9e3-7acea0d7ff80 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045149 Aug 08 02:29:07.041121 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 239/477] 10.222.0.27 () {58 vars in 1302 bytes} [Mon Aug 8 02:29:06 2022] GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:29:07.041284 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-50f4732a-2303-4dc9-b630-8b81a3f0f061 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:07.041284 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-50f4732a-2303-4dc9-b630-8b81a3f0f061 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:07.041284 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-50f4732a-2303-4dc9-b630-8b81a3f0f061 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:07.041284 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-50f4732a-2303-4dc9-b630-8b81a3f0f061 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:07.041625 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-50f4732a-2303-4dc9-b630-8b81a3f0f061 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:07.041625 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-50f4732a-2303-4dc9-b630-8b81a3f0f061 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:07.041625 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-50f4732a-2303-4dc9-b630-8b81a3f0f061 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:07.041625 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-50f4732a-2303-4dc9-b630-8b81a3f0f061 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:07.183620 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-50f4732a-2303-4dc9-b630-8b81a3f0f061 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:07.192437 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-50f4732a-2303-4dc9-b630-8b81a3f0f061 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1736 microversion: 2.1 time: 0.169812 Aug 08 02:29:07.193110 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 239/478] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:07 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1736 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:29:07.743299 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-bd5e5d10-a373-4fec-bfcc-1f3b1b54cb83 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:07.743917 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-bd5e5d10-a373-4fec-bfcc-1f3b1b54cb83 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:07.751771 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-bd5e5d10-a373-4fec-bfcc-1f3b1b54cb83 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:07.752133 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-bd5e5d10-a373-4fec-bfcc-1f3b1b54cb83 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:07.752575 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-bd5e5d10-a373-4fec-bfcc-1f3b1b54cb83 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:07.753321 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-bd5e5d10-a373-4fec-bfcc-1f3b1b54cb83 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:07.753880 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-bd5e5d10-a373-4fec-bfcc-1f3b1b54cb83 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:07.754312 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-bd5e5d10-a373-4fec-bfcc-1f3b1b54cb83 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:07.899220 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-bd5e5d10-a373-4fec-bfcc-1f3b1b54cb83 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:07.911199 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-bd5e5d10-a373-4fec-bfcc-1f3b1b54cb83 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1680 microversion: 2.1 time: 0.170852 Aug 08 02:29:07.911796 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 240/479] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:07 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1680 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:08.038377 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-245fece1-7ebb-461b-90b8-b32d503643b7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:08.040598 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-245fece1-7ebb-461b-90b8-b32d503643b7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: 95ae6b45-35d7-43a7-a021-91bb545cac4e] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:08.048736 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-245fece1-7ebb-461b-90b8-b32d503643b7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:08.049221 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-245fece1-7ebb-461b-90b8-b32d503643b7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:08.049651 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-245fece1-7ebb-461b-90b8-b32d503643b7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:08.050336 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-245fece1-7ebb-461b-90b8-b32d503643b7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:08.050784 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-245fece1-7ebb-461b-90b8-b32d503643b7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:08.051204 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-245fece1-7ebb-461b-90b8-b32d503643b7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:08.084335 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-245fece1-7ebb-461b-90b8-b32d503643b7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050042 Aug 08 02:29:08.085087 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 240/480] 10.222.0.27 () {58 vars in 1302 bytes} [Mon Aug 8 02:29:08 2022] GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:29:08.214628 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-1b177e2a-8bb4-4786-b495-bb7a405ae358 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:08.215532 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-1b177e2a-8bb4-4786-b495-bb7a405ae358 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:08.222766 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1b177e2a-8bb4-4786-b495-bb7a405ae358 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:08.223257 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1b177e2a-8bb4-4786-b495-bb7a405ae358 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:08.223691 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1b177e2a-8bb4-4786-b495-bb7a405ae358 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:08.224387 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1b177e2a-8bb4-4786-b495-bb7a405ae358 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:08.224881 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1b177e2a-8bb4-4786-b495-bb7a405ae358 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:08.225310 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1b177e2a-8bb4-4786-b495-bb7a405ae358 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:08.367103 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-1b177e2a-8bb4-4786-b495-bb7a405ae358 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:08.376293 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-1b177e2a-8bb4-4786-b495-bb7a405ae358 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1736 microversion: 2.1 time: 0.165491 Aug 08 02:29:08.377154 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 241/481] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:08 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1736 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:08.933129 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-58fb7e5c-ff69-4474-8487-0030efc4620e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:08.933744 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-58fb7e5c-ff69-4474-8487-0030efc4620e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:08.940816 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-58fb7e5c-ff69-4474-8487-0030efc4620e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:08.941306 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-58fb7e5c-ff69-4474-8487-0030efc4620e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:08.942024 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-58fb7e5c-ff69-4474-8487-0030efc4620e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:08.942695 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-58fb7e5c-ff69-4474-8487-0030efc4620e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:08.943144 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-58fb7e5c-ff69-4474-8487-0030efc4620e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:08.943568 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-58fb7e5c-ff69-4474-8487-0030efc4620e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:09.086929 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-58fb7e5c-ff69-4474-8487-0030efc4620e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:09.095611 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-58fb7e5c-ff69-4474-8487-0030efc4620e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1680 microversion: 2.1 time: 0.166638 Aug 08 02:29:09.096547 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 241/482] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:08 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1680 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:09.097022 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-a480b4b8-8426-42ea-be1b-622ee627f004 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:09.099219 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-a480b4b8-8426-42ea-be1b-622ee627f004 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: 95ae6b45-35d7-43a7-a021-91bb545cac4e] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:09.105511 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a480b4b8-8426-42ea-be1b-622ee627f004 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:09.106290 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a480b4b8-8426-42ea-be1b-622ee627f004 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:09.106881 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a480b4b8-8426-42ea-be1b-622ee627f004 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:09.107796 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a480b4b8-8426-42ea-be1b-622ee627f004 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:09.109089 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a480b4b8-8426-42ea-be1b-622ee627f004 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:09.109816 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a480b4b8-8426-42ea-be1b-622ee627f004 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:09.139445 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-a480b4b8-8426-42ea-be1b-622ee627f004 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.046560 Aug 08 02:29:09.139995 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 242/483] 10.222.0.27 () {58 vars in 1302 bytes} [Mon Aug 8 02:29:09 2022] GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e/os-volume_attachments => generated 25 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Aug 08 02:29:09.203013 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-e0eb635f-f262-4cf3-9681-142a66ebec37 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:09.203506 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-e0eb635f-f262-4cf3-9681-142a66ebec37 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: 95ae6b45-35d7-43a7-a021-91bb545cac4e] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:09.209588 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e0eb635f-f262-4cf3-9681-142a66ebec37 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:09.210081 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e0eb635f-f262-4cf3-9681-142a66ebec37 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:09.210512 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e0eb635f-f262-4cf3-9681-142a66ebec37 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:09.211194 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e0eb635f-f262-4cf3-9681-142a66ebec37 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:09.211643 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e0eb635f-f262-4cf3-9681-142a66ebec37 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:09.212066 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e0eb635f-f262-4cf3-9681-142a66ebec37 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:09.237101 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-e0eb635f-f262-4cf3-9681-142a66ebec37 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: 95ae6b45-35d7-43a7-a021-91bb545cac4e] Going to try to terminate instance {{(pid=118352) delete /opt/stack/nova/nova/compute/api.py:2666}} Aug 08 02:29:09.398893 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-a64000ca-5c40-47d6-bb92-749e3e531217 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:09.399511 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-a64000ca-5c40-47d6-bb92-749e3e531217 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:09.405936 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a64000ca-5c40-47d6-bb92-749e3e531217 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:09.406438 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a64000ca-5c40-47d6-bb92-749e3e531217 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:09.406880 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a64000ca-5c40-47d6-bb92-749e3e531217 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:09.407574 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a64000ca-5c40-47d6-bb92-749e3e531217 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:09.408033 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a64000ca-5c40-47d6-bb92-749e3e531217 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:09.408462 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a64000ca-5c40-47d6-bb92-749e3e531217 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:09.542474 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-a64000ca-5c40-47d6-bb92-749e3e531217 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:09.552735 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-a64000ca-5c40-47d6-bb92-749e3e531217 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1736 microversion: 2.1 time: 0.157566 Aug 08 02:29:09.554349 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 243/484] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:09 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1736 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:29:09.625653 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-e0eb635f-f262-4cf3-9681-142a66ebec37 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "DELETE /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e" status: 204 len: 0 microversion: 2.1 time: 0.426313 Aug 08 02:29:09.626370 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 242/485] 10.222.0.27 () {58 vars in 1239 bytes} [Mon Aug 8 02:29:09 2022] DELETE /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e => generated 0 bytes in 428 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 08 02:29:09.634604 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-58451cb2-2e38-418c-98b0-7f302e65ce84 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:09.635218 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-58451cb2-2e38-418c-98b0-7f302e65ce84 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: 95ae6b45-35d7-43a7-a021-91bb545cac4e] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:09.641506 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-58451cb2-2e38-418c-98b0-7f302e65ce84 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:09.641989 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-58451cb2-2e38-418c-98b0-7f302e65ce84 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:09.642426 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-58451cb2-2e38-418c-98b0-7f302e65ce84 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:09.643115 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-58451cb2-2e38-418c-98b0-7f302e65ce84 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:09.643579 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-58451cb2-2e38-418c-98b0-7f302e65ce84 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:09.644003 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-58451cb2-2e38-418c-98b0-7f302e65ce84 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:09.784748 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-58451cb2-2e38-418c-98b0-7f302e65ce84 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:09.792210 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-58451cb2-2e38-418c-98b0-7f302e65ce84 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e" status: 200 len: 1682 microversion: 2.1 time: 0.161505 Aug 08 02:29:09.792964 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 244/486] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:09 2022] GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e => generated 1682 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:10.118179 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-f47349ad-91b9-41a4-845e-9e1c82045248 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:10.120000 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-f47349ad-91b9-41a4-845e-9e1c82045248 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:10.127750 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f47349ad-91b9-41a4-845e-9e1c82045248 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:10.128347 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f47349ad-91b9-41a4-845e-9e1c82045248 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:10.129070 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f47349ad-91b9-41a4-845e-9e1c82045248 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:10.129858 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f47349ad-91b9-41a4-845e-9e1c82045248 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:10.130407 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f47349ad-91b9-41a4-845e-9e1c82045248 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:10.130937 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f47349ad-91b9-41a4-845e-9e1c82045248 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:10.268029 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-f47349ad-91b9-41a4-845e-9e1c82045248 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:10.275510 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-f47349ad-91b9-41a4-845e-9e1c82045248 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1657 microversion: 2.1 time: 0.161983 Aug 08 02:29:10.276071 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 243/487] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:10 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1657 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:10.295739 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-3a53346c-3359-42ea-8bff-cc781e788b04 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:29:10.301964 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3a53346c-3359-42ea-8bff-cc781e788b04 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:10.302504 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3a53346c-3359-42ea-8bff-cc781e788b04 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:10.302951 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3a53346c-3359-42ea-8bff-cc781e788b04 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:10.325624 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-3a53346c-3359-42ea-8bff-cc781e788b04 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Going to try to start instance {{(pid=118351) start /opt/stack/nova/nova/compute/api.py:2732}} Aug 08 02:29:10.525324 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-3a53346c-3359-42ea-8bff-cc781e788b04 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "POST /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a/action" status: 202 len: 0 microversion: 2.1 time: 0.233361 Aug 08 02:29:10.526009 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 245/488] 10.222.0.27 () {60 vars in 1278 bytes} [Mon Aug 8 02:29:10 2022] POST /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a/action => generated 0 bytes in 235 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 08 02:29:10.535531 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-a31431c2-aa18-4e42-b163-08b31fe792d2 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:10.536258 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-a31431c2-aa18-4e42-b163-08b31fe792d2 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:10.545386 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a31431c2-aa18-4e42-b163-08b31fe792d2 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:10.547104 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a31431c2-aa18-4e42-b163-08b31fe792d2 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:10.547684 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a31431c2-aa18-4e42-b163-08b31fe792d2 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:10.548548 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a31431c2-aa18-4e42-b163-08b31fe792d2 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:10.549357 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a31431c2-aa18-4e42-b163-08b31fe792d2 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:10.550139 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a31431c2-aa18-4e42-b163-08b31fe792d2 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:10.575683 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-8c4c57dd-b6a1-4e10-aa20-30e1a8b5e110 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:10.576884 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-8c4c57dd-b6a1-4e10-aa20-30e1a8b5e110 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:10.583201 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8c4c57dd-b6a1-4e10-aa20-30e1a8b5e110 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:10.583687 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8c4c57dd-b6a1-4e10-aa20-30e1a8b5e110 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:10.584117 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8c4c57dd-b6a1-4e10-aa20-30e1a8b5e110 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:10.584841 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8c4c57dd-b6a1-4e10-aa20-30e1a8b5e110 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:10.585293 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8c4c57dd-b6a1-4e10-aa20-30e1a8b5e110 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:10.585717 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8c4c57dd-b6a1-4e10-aa20-30e1a8b5e110 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:10.738761 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-a31431c2-aa18-4e42-b163-08b31fe792d2 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:10.746418 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-a31431c2-aa18-4e42-b163-08b31fe792d2 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1666 microversion: 2.1 time: 0.215823 Aug 08 02:29:10.746988 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 244/489] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:10 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1666 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:10.766239 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-8c4c57dd-b6a1-4e10-aa20-30e1a8b5e110 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:10.774202 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-8c4c57dd-b6a1-4e10-aa20-30e1a8b5e110 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1736 microversion: 2.1 time: 0.203385 Aug 08 02:29:10.774771 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 246/490] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:10 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1736 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:29:10.815694 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-f692783a-784a-4115-a94a-0c77aa67b43e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:10.816235 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-f692783a-784a-4115-a94a-0c77aa67b43e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: 95ae6b45-35d7-43a7-a021-91bb545cac4e] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:10.822642 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f692783a-784a-4115-a94a-0c77aa67b43e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:10.823137 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f692783a-784a-4115-a94a-0c77aa67b43e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:10.823608 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f692783a-784a-4115-a94a-0c77aa67b43e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:10.824299 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f692783a-784a-4115-a94a-0c77aa67b43e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:10.824809 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f692783a-784a-4115-a94a-0c77aa67b43e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:10.825239 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f692783a-784a-4115-a94a-0c77aa67b43e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:10.961299 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-f692783a-784a-4115-a94a-0c77aa67b43e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:10.969461 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-f692783a-784a-4115-a94a-0c77aa67b43e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e" status: 200 len: 1682 microversion: 2.1 time: 0.158830 Aug 08 02:29:10.972427 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 245/491] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:10 2022] GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e => generated 1682 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:11.769287 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-8fdfbd18-4d90-4d61-b4b5-c27812e49ad2 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:11.769896 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-8fdfbd18-4d90-4d61-b4b5-c27812e49ad2 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:11.776257 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8fdfbd18-4d90-4d61-b4b5-c27812e49ad2 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:11.776774 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8fdfbd18-4d90-4d61-b4b5-c27812e49ad2 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:11.777269 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8fdfbd18-4d90-4d61-b4b5-c27812e49ad2 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:11.777955 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8fdfbd18-4d90-4d61-b4b5-c27812e49ad2 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:11.778409 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8fdfbd18-4d90-4d61-b4b5-c27812e49ad2 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:11.778892 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8fdfbd18-4d90-4d61-b4b5-c27812e49ad2 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:11.794951 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-357fd65e-10da-498f-987d-16e0dd90534a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:11.795675 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-357fd65e-10da-498f-987d-16e0dd90534a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:11.802356 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-357fd65e-10da-498f-987d-16e0dd90534a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:11.802843 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-357fd65e-10da-498f-987d-16e0dd90534a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:11.803272 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-357fd65e-10da-498f-987d-16e0dd90534a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:11.803953 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-357fd65e-10da-498f-987d-16e0dd90534a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:11.804402 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-357fd65e-10da-498f-987d-16e0dd90534a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:11.804850 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-357fd65e-10da-498f-987d-16e0dd90534a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:11.981666 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-8fdfbd18-4d90-4d61-b4b5-c27812e49ad2 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:11.989610 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-8fdfbd18-4d90-4d61-b4b5-c27812e49ad2 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1666 microversion: 2.1 time: 0.224828 Aug 08 02:29:11.990180 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 247/492] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:11 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1666 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:11.995869 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-a09fd7a1-e01c-40f1-8469-68bdd6e68d66 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:11.996719 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-a09fd7a1-e01c-40f1-8469-68bdd6e68d66 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: 95ae6b45-35d7-43a7-a021-91bb545cac4e] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:12.003474 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a09fd7a1-e01c-40f1-8469-68bdd6e68d66 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:12.004201 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a09fd7a1-e01c-40f1-8469-68bdd6e68d66 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:12.004845 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a09fd7a1-e01c-40f1-8469-68bdd6e68d66 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:12.009829 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a09fd7a1-e01c-40f1-8469-68bdd6e68d66 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:12.009829 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a09fd7a1-e01c-40f1-8469-68bdd6e68d66 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:12.009829 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a09fd7a1-e01c-40f1-8469-68bdd6e68d66 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:12.015500 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-357fd65e-10da-498f-987d-16e0dd90534a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:12.028725 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-357fd65e-10da-498f-987d-16e0dd90534a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1736 microversion: 2.1 time: 0.236981 Aug 08 02:29:12.029605 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 246/493] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:11 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1736 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:29:12.149685 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-a09fd7a1-e01c-40f1-8469-68bdd6e68d66 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:12.157800 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-a09fd7a1-e01c-40f1-8469-68bdd6e68d66 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e" status: 200 len: 1682 microversion: 2.1 time: 0.166310 Aug 08 02:29:12.158485 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 248/494] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:11 2022] GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e => generated 1682 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:13.011156 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-c833a2b9-b220-411a-925f-d16124c0ea47 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:13.015163 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-c833a2b9-b220-411a-925f-d16124c0ea47 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:13.019458 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c833a2b9-b220-411a-925f-d16124c0ea47 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:13.019939 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c833a2b9-b220-411a-925f-d16124c0ea47 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:13.020372 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c833a2b9-b220-411a-925f-d16124c0ea47 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:13.021087 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c833a2b9-b220-411a-925f-d16124c0ea47 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:13.021695 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c833a2b9-b220-411a-925f-d16124c0ea47 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:13.022268 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c833a2b9-b220-411a-925f-d16124c0ea47 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:13.051426 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-3f732f2f-6705-4b63-bc35-7b9e521ead08 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:13.054714 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-3f732f2f-6705-4b63-bc35-7b9e521ead08 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:13.067462 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3f732f2f-6705-4b63-bc35-7b9e521ead08 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:13.072530 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3f732f2f-6705-4b63-bc35-7b9e521ead08 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:13.072530 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3f732f2f-6705-4b63-bc35-7b9e521ead08 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:13.073183 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3f732f2f-6705-4b63-bc35-7b9e521ead08 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:13.073978 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3f732f2f-6705-4b63-bc35-7b9e521ead08 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:13.074503 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3f732f2f-6705-4b63-bc35-7b9e521ead08 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:13.222581 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-c833a2b9-b220-411a-925f-d16124c0ea47 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:13.232117 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-c833a2b9-b220-411a-925f-d16124c0ea47 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1666 microversion: 2.1 time: 0.224767 Aug 08 02:29:13.233033 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 247/495] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:13 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1666 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:13.237805 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-72158a74-30a1-4ea8-b4d2-6a834d281279 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:13.238610 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-72158a74-30a1-4ea8-b4d2-6a834d281279 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: 95ae6b45-35d7-43a7-a021-91bb545cac4e] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:13.245724 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-72158a74-30a1-4ea8-b4d2-6a834d281279 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:13.246439 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-72158a74-30a1-4ea8-b4d2-6a834d281279 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:13.247045 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-72158a74-30a1-4ea8-b4d2-6a834d281279 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:13.249294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-72158a74-30a1-4ea8-b4d2-6a834d281279 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:13.250325 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-72158a74-30a1-4ea8-b4d2-6a834d281279 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:13.250795 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-72158a74-30a1-4ea8-b4d2-6a834d281279 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:13.272092 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-3f732f2f-6705-4b63-bc35-7b9e521ead08 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:13.282099 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-3f732f2f-6705-4b63-bc35-7b9e521ead08 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1736 microversion: 2.1 time: 0.235050 Aug 08 02:29:13.282682 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 249/496] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:13 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1736 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:13.342781 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-72158a74-30a1-4ea8-b4d2-6a834d281279 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:13.350698 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-72158a74-30a1-4ea8-b4d2-6a834d281279 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e" status: 200 len: 1615 microversion: 2.1 time: 0.116410 Aug 08 02:29:13.351254 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 248/497] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:13 2022] GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e => generated 1615 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:13.374372 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [req-4d0e71ea-eed9-4ff4-b610-1d058d567717 req-f33b35f8-2f1e-4aa9-8116-bf445aa50f83 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "95ae6b45-35d7-43a7-a021-91bb545cac4e", "name": "network-vif-deleted", "tag": "26e540df-eae8-4a69-873e-d4b1a210cfdf"}]} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:29:13.383383 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-4d0e71ea-eed9-4ff4-b610-1d058d567717 req-f33b35f8-2f1e-4aa9-8116-bf445aa50f83 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:13.384080 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-4d0e71ea-eed9-4ff4-b610-1d058d567717 req-f33b35f8-2f1e-4aa9-8116-bf445aa50f83 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:13.384724 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-4d0e71ea-eed9-4ff4-b610-1d058d567717 req-f33b35f8-2f1e-4aa9-8116-bf445aa50f83 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:13.397002 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.compute.server_external_events [req-4d0e71ea-eed9-4ff4-b610-1d058d567717 req-f33b35f8-2f1e-4aa9-8116-bf445aa50f83 service nova] Creating event network-vif-deleted:26e540df-eae8-4a69-873e-d4b1a210cfdf for instance 95ae6b45-35d7-43a7-a021-91bb545cac4e on hv-cinder-74014 Aug 08 02:29:13.402270 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [req-4d0e71ea-eed9-4ff4-b610-1d058d567717 req-f33b35f8-2f1e-4aa9-8116-bf445aa50f83 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033371 Aug 08 02:29:13.402864 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 250/498] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:29:13 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:29:14.254357 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-ee15eac1-27f6-47c3-8878-6d0632d02b70 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:14.255433 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-ee15eac1-27f6-47c3-8878-6d0632d02b70 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:14.261295 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ee15eac1-27f6-47c3-8878-6d0632d02b70 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:14.261771 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ee15eac1-27f6-47c3-8878-6d0632d02b70 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:14.262251 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ee15eac1-27f6-47c3-8878-6d0632d02b70 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:14.262929 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ee15eac1-27f6-47c3-8878-6d0632d02b70 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:14.263380 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ee15eac1-27f6-47c3-8878-6d0632d02b70 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:14.263798 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ee15eac1-27f6-47c3-8878-6d0632d02b70 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:14.305713 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-1de8a84b-8b5c-4c43-8057-fc4c6b5875a4 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:14.306674 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-1de8a84b-8b5c-4c43-8057-fc4c6b5875a4 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:14.314532 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1de8a84b-8b5c-4c43-8057-fc4c6b5875a4 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:14.315170 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1de8a84b-8b5c-4c43-8057-fc4c6b5875a4 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:14.315733 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1de8a84b-8b5c-4c43-8057-fc4c6b5875a4 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:14.317486 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1de8a84b-8b5c-4c43-8057-fc4c6b5875a4 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:14.318194 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1de8a84b-8b5c-4c43-8057-fc4c6b5875a4 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:14.318859 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1de8a84b-8b5c-4c43-8057-fc4c6b5875a4 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:14.466088 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-ee15eac1-27f6-47c3-8878-6d0632d02b70 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:14.475294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-ee15eac1-27f6-47c3-8878-6d0632d02b70 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1670 microversion: 2.1 time: 0.225668 Aug 08 02:29:14.489002 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 249/499] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:14 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1670 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:14.489002 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-87e3d342-964e-4c25-b2b1-e1fbcc549b5c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:14.489002 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-87e3d342-964e-4c25-b2b1-e1fbcc549b5c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: 95ae6b45-35d7-43a7-a021-91bb545cac4e] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:14.499693 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-87e3d342-964e-4c25-b2b1-e1fbcc549b5c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:14.501198 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-87e3d342-964e-4c25-b2b1-e1fbcc549b5c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:14.501614 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-87e3d342-964e-4c25-b2b1-e1fbcc549b5c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:14.502327 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-87e3d342-964e-4c25-b2b1-e1fbcc549b5c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:14.502793 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-87e3d342-964e-4c25-b2b1-e1fbcc549b5c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:14.503322 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-87e3d342-964e-4c25-b2b1-e1fbcc549b5c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:14.516031 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-1de8a84b-8b5c-4c43-8057-fc4c6b5875a4 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:14.524736 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-1de8a84b-8b5c-4c43-8057-fc4c6b5875a4 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1736 microversion: 2.1 time: 0.223949 Aug 08 02:29:14.525350 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 251/500] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:14 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1736 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:14.537543 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-87e3d342-964e-4c25-b2b1-e1fbcc549b5c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:14.538211 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-87e3d342-964e-4c25-b2b1-e1fbcc549b5c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:14.538655 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-87e3d342-964e-4c25-b2b1-e1fbcc549b5c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:14.542979 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.objects.instance [None req-87e3d342-964e-4c25-b2b1-e1fbcc549b5c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lazy-loading 'fault' on Instance uuid 95ae6b45-35d7-43a7-a021-91bb545cac4e {{(pid=118352) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 08 02:29:14.583287 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-87e3d342-964e-4c25-b2b1-e1fbcc549b5c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:14.592842 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-87e3d342-964e-4c25-b2b1-e1fbcc549b5c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e" status: 200 len: 1331 microversion: 2.1 time: 0.115879 Aug 08 02:29:14.597322 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 250/501] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:14 2022] GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e => generated 1331 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:15.093328 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-4e69b1ac-15ee-4a8f-9a4c-023fd2a35a87 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "14f9aecb-ce30-4121-87dd-237586a72402", "device": "/dev/sdb"}} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:29:15.096821 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-4e69b1ac-15ee-4a8f-9a4c-023fd2a35a87 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: f2edabed-f896-4ecb-ab34-ef137c01d581] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:15.098832 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_api:SELECT=97,nova_api:UPDATE=1,nova_api:INSERT=1,nova_api:DELETE=1 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:29:15.103543 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4e69b1ac-15ee-4a8f-9a4c-023fd2a35a87 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:15.104240 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4e69b1ac-15ee-4a8f-9a4c-023fd2a35a87 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:15.104458 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4e69b1ac-15ee-4a8f-9a4c-023fd2a35a87 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:15.105177 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4e69b1ac-15ee-4a8f-9a4c-023fd2a35a87 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:15.105632 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4e69b1ac-15ee-4a8f-9a4c-023fd2a35a87 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:15.106057 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4e69b1ac-15ee-4a8f-9a4c-023fd2a35a87 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:15.109781 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:29:15.550248 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-62e864c4-d59a-4a60-b891-2b12f9b77532 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:15.550248 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-62e864c4-d59a-4a60-b891-2b12f9b77532 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:15.554075 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-62e864c4-d59a-4a60-b891-2b12f9b77532 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:15.554681 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-62e864c4-d59a-4a60-b891-2b12f9b77532 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:15.555378 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-62e864c4-d59a-4a60-b891-2b12f9b77532 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:15.556180 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-62e864c4-d59a-4a60-b891-2b12f9b77532 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:15.556907 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-62e864c4-d59a-4a60-b891-2b12f9b77532 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:15.557428 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-62e864c4-d59a-4a60-b891-2b12f9b77532 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:15.926807 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-62e864c4-d59a-4a60-b891-2b12f9b77532 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:15.936531 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-62e864c4-d59a-4a60-b891-2b12f9b77532 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1736 microversion: 2.1 time: 0.394284 Aug 08 02:29:15.937167 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 251/502] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:15 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1736 bytes in 396 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:15.955648 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-1ceca6b5-df10-464e-90cb-a1b9667b63d6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:15.956266 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-1ceca6b5-df10-464e-90cb-a1b9667b63d6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: 95ae6b45-35d7-43a7-a021-91bb545cac4e] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:15.958237 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_api:SELECT=95,nova_api:UPDATE=1 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:29:15.962906 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1ceca6b5-df10-464e-90cb-a1b9667b63d6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:15.963392 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1ceca6b5-df10-464e-90cb-a1b9667b63d6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:15.963852 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1ceca6b5-df10-464e-90cb-a1b9667b63d6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:15.964555 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1ceca6b5-df10-464e-90cb-a1b9667b63d6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:15.965049 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1ceca6b5-df10-464e-90cb-a1b9667b63d6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:15.965510 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1ceca6b5-df10-464e-90cb-a1b9667b63d6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:15.969613 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:29:15.996948 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1ceca6b5-df10-464e-90cb-a1b9667b63d6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:15.997434 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1ceca6b5-df10-464e-90cb-a1b9667b63d6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:15.997868 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1ceca6b5-df10-464e-90cb-a1b9667b63d6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:15.998319 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.objects.instance [None req-1ceca6b5-df10-464e-90cb-a1b9667b63d6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lazy-loading 'fault' on Instance uuid 95ae6b45-35d7-43a7-a021-91bb545cac4e {{(pid=118352) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 08 02:29:16.038186 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-1ceca6b5-df10-464e-90cb-a1b9667b63d6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:16.045935 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-1ceca6b5-df10-464e-90cb-a1b9667b63d6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e" status: 200 len: 1331 microversion: 2.1 time: 0.094077 Aug 08 02:29:16.060142 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 252/503] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:15 2022] GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e => generated 1331 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:16.173574 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-4e69b1ac-15ee-4a8f-9a4c-023fd2a35a87 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "POST /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.084012 Aug 08 02:29:16.174359 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 252/504] 10.222.0.27 () {60 vars in 1323 bytes} [Mon Aug 8 02:29:15 2022] POST /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581/os-volume_attachments => generated 194 bytes in 1086 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:29:16.959654 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-5693ef67-4200-4f54-9c78-a68f6fd1e5b1 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:16.960303 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-5693ef67-4200-4f54-9c78-a68f6fd1e5b1 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:16.968059 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5693ef67-4200-4f54-9c78-a68f6fd1e5b1 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:16.968559 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5693ef67-4200-4f54-9c78-a68f6fd1e5b1 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:16.969176 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5693ef67-4200-4f54-9c78-a68f6fd1e5b1 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:16.969863 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5693ef67-4200-4f54-9c78-a68f6fd1e5b1 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:16.970319 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5693ef67-4200-4f54-9c78-a68f6fd1e5b1 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:16.970743 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5693ef67-4200-4f54-9c78-a68f6fd1e5b1 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:17.071782 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-12de098b-07e6-4cd2-acc2-0a6654b61955 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:17.072573 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-12de098b-07e6-4cd2-acc2-0a6654b61955 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: 95ae6b45-35d7-43a7-a021-91bb545cac4e] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:17.080300 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-12de098b-07e6-4cd2-acc2-0a6654b61955 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:17.080871 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-12de098b-07e6-4cd2-acc2-0a6654b61955 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:17.081305 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-12de098b-07e6-4cd2-acc2-0a6654b61955 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:17.082154 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-12de098b-07e6-4cd2-acc2-0a6654b61955 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:17.082602 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-12de098b-07e6-4cd2-acc2-0a6654b61955 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:17.083023 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-12de098b-07e6-4cd2-acc2-0a6654b61955 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:17.097129 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.wsgi [None req-12de098b-07e6-4cd2-acc2-0a6654b61955 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] HTTP exception thrown: Instance 95ae6b45-35d7-43a7-a021-91bb545cac4e could not be found. Aug 08 02:29:17.098747 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-12de098b-07e6-4cd2-acc2-0a6654b61955 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Request method failure captured: Aug 08 02:29:17.098747 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: request: GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e HTTP/1.1 Aug 08 02:29:17.098747 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Accept: application/json Aug 08 02:29:17.098747 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Accept-Encoding: identity Aug 08 02:29:17.098747 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Connection: close Aug 08 02:29:17.098747 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Content-Length: 0 Aug 08 02:29:17.098747 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Content-Type: application/json Aug 08 02:29:17.098747 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Host: 10.222.0.27 Aug 08 02:29:17.098747 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Openstack-System-Scope: None Aug 08 02:29:17.098747 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: User-Agent: python-urllib3/1.26.11 Aug 08 02:29:17.098747 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Auth-Token: gAAAAABi8HQUs7QI_wjv0vSDVlijMnZ9-Lkgqc--Bu80jwWL-CoTBjbpe1sae_INrRIGeFqgwK-ezXnEOgqpG2NO5P97SEgPcY4zY_QgnaTA6xv-rzef5kDF5s4pU5M8QdM-T8eUF020_v3zsmgR78rxXaglg0C5QuxYvT8R4THTJZuwa_f26Bo Aug 08 02:29:17.098747 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Domain-Id: None Aug 08 02:29:17.098747 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Domain-Name: None Aug 08 02:29:17.098747 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Identity-Status: Confirmed Aug 08 02:29:17.098747 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Is-Admin-Project: True Aug 08 02:29:17.098747 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Domain-Id: default Aug 08 02:29:17.098747 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Domain-Name: Default Aug 08 02:29:17.098747 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Id: ca29b2b9a59d4826bfd25f4dcfa9cffe Aug 08 02:29:17.098747 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Name: tempest-AttachVolumeNegativeTest-420450256 Aug 08 02:29:17.098747 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Role: reader,member Aug 08 02:29:17.098747 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Roles: reader,member Aug 08 02:29:17.100361 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/ca29b2b9a59d4826bfd25f4dcfa9cffe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/ca29b2b9a59d4826bfd25f4dcfa9cffe"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_ca29b2b9a59d4826bfd25f4dcfa9cffe", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/ca29b2b9a59d4826bfd25f4dcfa9cffe"}]}] Aug 08 02:29:17.100361 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant: tempest-AttachVolumeNegativeTest-420450256 Aug 08 02:29:17.100361 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant-Id: ca29b2b9a59d4826bfd25f4dcfa9cffe Aug 08 02:29:17.100361 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-420450256 Aug 08 02:29:17.100361 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User: tempest-AttachVolumeNegativeTest-420450256-project Aug 08 02:29:17.100361 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Domain-Id: default Aug 08 02:29:17.100361 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Domain-Name: Default Aug 08 02:29:17.100361 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Id: 578cd837d9fe442587f8c6c8528334c0 Aug 08 02:29:17.100361 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Name: tempest-AttachVolumeNegativeTest-420450256-project Aug 08 02:29:17.100361 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: method: > Aug 08 02:29:17.100361 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: exception: Instance 95ae6b45-35d7-43a7-a021-91bb545cac4e could not be found. Aug 08 02:29:17.100361 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: action_args: {'id': '95ae6b45-35d7-43a7-a021-91bb545cac4e'} Aug 08 02:29:17.105304 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:29:17.105304 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:29:17.105304 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 08 02:29:17.105304 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 08 02:29:17.105304 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 08 02:29:17.105304 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 08 02:29:17.105304 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 08 02:29:17.105304 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 08 02:29:17.105304 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 08 02:29:17.105304 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 08 02:29:17.105304 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 95ae6b45-35d7-43a7-a021-91bb545cac4e could not be found. Aug 08 02:29:17.105304 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:29:17.105304 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:29:17.105304 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:29:17.105304 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:29:17.105304 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:29:17.105304 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:29:17.105304 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:29:17.105304 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:29:17.105304 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:29:17.106603 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:29:17.106603 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 08 02:29:17.106603 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 08 02:29:17.106603 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 08 02:29:17.106603 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 08 02:29:17.106603 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 08 02:29:17.106603 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:29:17.106603 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 95ae6b45-35d7-43a7-a021-91bb545cac4e could not be found. Aug 08 02:29:17.106603 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:29:17.106603 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:29:17.106603 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:29:17.106603 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:29:17.106603 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:29:17.106603 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:29:17.106603 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:29:17.106603 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:29:17.106603 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 95ae6b45-35d7-43a7-a021-91bb545cac4e could not be found. Aug 08 02:29:17.106603 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:29:17.107977 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-12de098b-07e6-4cd2-acc2-0a6654b61955 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Returning 404 to user: Instance 95ae6b45-35d7-43a7-a021-91bb545cac4e could not be found. {{(pid=118351) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:29:17.107977 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-12de098b-07e6-4cd2-acc2-0a6654b61955 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e" status: 404 len: 111 microversion: 2.1 time: 0.033071 Aug 08 02:29:17.107977 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 253/505] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:17 2022] GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Aug 08 02:29:17.108475 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-5693ef67-4200-4f54-9c78-a68f6fd1e5b1 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:17.108815 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-dad7f130-cd08-41f4-a466-804605f4ff2c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '.version_select at 0x7efc761bdf70>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:17.111268 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-5693ef67-4200-4f54-9c78-a68f6fd1e5b1 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1736 microversion: 2.1 time: 0.155632 Aug 08 02:29:17.111843 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 253/506] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:16 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1736 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:17.214456 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-dad7f130-cd08-41f4-a466-804605f4ff2c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1410729274" status: 202 len: 0 microversion: 2.1 time: 0.109062 Aug 08 02:29:17.215042 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 254/507] 10.222.0.27 () {58 vars in 1221 bytes} [Mon Aug 8 02:29:17 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1410729274 => generated 0 bytes in 110 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Aug 08 02:29:18.004242 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-80eb7676-719e-44b1-9a37-2a663d1a8b8e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Action: 'create', calling method: .version_select at 0x7efc74594700>, body: {"keypair": {"name": "tempest-keypair-1247410672", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNu2nDzYV3P6tBmd9wZijSImpLmIPk1P9ciPNW8mTrfz4CbcorDWlOtVLGF/QN1V3BMEpaC0xFDyw+ZClEFFfed0iOtkKNpOt86YF5csI3zYQLPMneI+guYZ+bwcx84HHQ=="}} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:29:18.023393 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.quota [None req-80eb7676-719e-44b1-9a37-2a663d1a8b8e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Getting quotas for project ca29b2b9a59d4826bfd25f4dcfa9cffe. Resources: {'key_pairs'} {{(pid=118352) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 08 02:29:18.024916 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.quota [None req-80eb7676-719e-44b1-9a37-2a663d1a8b8e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Getting quotas for user 578cd837d9fe442587f8c6c8528334c0 and project ca29b2b9a59d4826bfd25f4dcfa9cffe. Resources: {'key_pairs'} {{(pid=118352) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 08 02:29:18.132945 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-9affc194-54c7-4a3f-994e-808843902ab0 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:18.133593 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-9affc194-54c7-4a3f-994e-808843902ab0 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:18.140538 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-9affc194-54c7-4a3f-994e-808843902ab0 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:18.141059 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-9affc194-54c7-4a3f-994e-808843902ab0 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:18.141999 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-9affc194-54c7-4a3f-994e-808843902ab0 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:18.142774 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-9affc194-54c7-4a3f-994e-808843902ab0 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:18.143252 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-9affc194-54c7-4a3f-994e-808843902ab0 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:18.143669 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-9affc194-54c7-4a3f-994e-808843902ab0 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:18.158524 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-80eb7676-719e-44b1-9a37-2a663d1a8b8e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.158391 Aug 08 02:29:18.161358 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 254/508] 10.222.0.27 () {60 vars in 1159 bytes} [Mon Aug 8 02:29:17 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:29:18.287372 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-9affc194-54c7-4a3f-994e-808843902ab0 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:18.295424 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-9affc194-54c7-4a3f-994e-808843902ab0 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1716 microversion: 2.1 time: 0.167104 Aug 08 02:29:18.301079 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 255/509] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:18 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1716 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:29:18.410193 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-df018152-dfbe-4c76-9255-f069d6be6ed4 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:18.410824 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-df018152-dfbe-4c76-9255-f069d6be6ed4 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:18.417927 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-df018152-dfbe-4c76-9255-f069d6be6ed4 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:18.418446 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-df018152-dfbe-4c76-9255-f069d6be6ed4 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:18.418892 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-df018152-dfbe-4c76-9255-f069d6be6ed4 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:18.419687 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-df018152-dfbe-4c76-9255-f069d6be6ed4 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:18.420205 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-df018152-dfbe-4c76-9255-f069d6be6ed4 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:18.420918 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-df018152-dfbe-4c76-9255-f069d6be6ed4 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:18.505673 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.wsgi [None req-df018152-dfbe-4c76-9255-f069d6be6ed4 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] HTTP exception thrown: Cannot 'detach_volume' instance a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 while it is in vm_state rescued Aug 08 02:29:18.507295 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-df018152-dfbe-4c76-9255-f069d6be6ed4 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Request method failure captured: Aug 08 02:29:18.507295 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: request: DELETE /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24/os-volume_attachments/7cdab395-b182-49c3-8bf5-3e6673f0dc52 HTTP/1.1 Aug 08 02:29:18.507295 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept: application/json Aug 08 02:29:18.507295 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept-Encoding: identity Aug 08 02:29:18.507295 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Connection: close Aug 08 02:29:18.507295 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Length: 0 Aug 08 02:29:18.507295 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Type: application/json Aug 08 02:29:18.507295 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Host: 10.222.0.27 Aug 08 02:29:18.507295 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Openstack-System-Scope: None Aug 08 02:29:18.507295 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: User-Agent: python-urllib3/1.26.11 Aug 08 02:29:18.507295 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Auth-Token: gAAAAABi8HQXsGK5SLdeRHabROI8DgUVfT8-qt3OmjigKWIwr2r9ylfDom5hOf3PBwXXbVpQKaQQCg2GWwYCANhQNXkemxQtH2Z8vABNoiUbOWpRK-0iPQcLUey3hJs3m48mZ7E_Wt-sxx-vNwT0dxN6KL1UanLlX0GY-h9OUcxTwCxsePW3BgQ Aug 08 02:29:18.507295 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Id: None Aug 08 02:29:18.507295 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Name: None Aug 08 02:29:18.507295 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Identity-Status: Confirmed Aug 08 02:29:18.507295 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Is-Admin-Project: True Aug 08 02:29:18.507295 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Id: default Aug 08 02:29:18.507295 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Name: Default Aug 08 02:29:18.507295 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Id: 8af7d2bbc6d94273bd39c81423d8cebe Aug 08 02:29:18.507295 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-514896148 Aug 08 02:29:18.507295 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Role: reader,member Aug 08 02:29:18.507295 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Roles: reader,member Aug 08 02:29:18.509050 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/8af7d2bbc6d94273bd39c81423d8cebe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/8af7d2bbc6d94273bd39c81423d8cebe"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_8af7d2bbc6d94273bd39c81423d8cebe", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/8af7d2bbc6d94273bd39c81423d8cebe"}]}] Aug 08 02:29:18.509050 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant: tempest-ServerRescueNegativeTestJSON-514896148 Aug 08 02:29:18.509050 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Id: 8af7d2bbc6d94273bd39c81423d8cebe Aug 08 02:29:18.509050 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-514896148 Aug 08 02:29:18.509050 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User: tempest-ServerRescueNegativeTestJSON-514896148-project Aug 08 02:29:18.509050 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Id: default Aug 08 02:29:18.509050 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Name: Default Aug 08 02:29:18.509050 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Id: 5c8ae69712c14878bd2aab6d33512b7a Aug 08 02:29:18.509050 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Name: tempest-ServerRescueNegativeTestJSON-514896148-project Aug 08 02:29:18.509050 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: method: > Aug 08 02:29:18.509050 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: exception: Cannot 'detach_volume' instance a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 while it is in vm_state rescued Aug 08 02:29:18.512897 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: action_args: {'server_id': 'a5b1155c-5bba-46ae-8f3c-61f70c8a8a24', 'id': '7cdab395-b182-49c3-8bf5-3e6673f0dc52'} Aug 08 02:29:18.512897 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:29:18.512897 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:29:18.512897 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 08 02:29:18.512897 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 08 02:29:18.512897 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 08 02:29:18.512897 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 08 02:29:18.512897 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner Aug 08 02:29:18.512897 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Aug 08 02:29:18.512897 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 in vm_state rescued. Cannot detach_volume while the instance is in this state. Aug 08 02:29:18.512897 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:29:18.512897 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:29:18.512897 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:29:18.512897 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:29:18.512897 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:29:18.512897 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:29:18.512897 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:29:18.512897 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:29:18.512897 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:29:18.512897 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:29:18.514324 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete Aug 08 02:29:18.514324 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Aug 08 02:29:18.514324 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Aug 08 02:29:18.514324 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Aug 08 02:29:18.514324 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 while it is in vm_state rescued Aug 08 02:29:18.514324 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:29:18.514324 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:29:18.514324 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:29:18.514324 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:29:18.514324 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:29:18.514324 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:29:18.514324 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:29:18.514324 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:29:18.514324 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 while it is in vm_state rescued Aug 08 02:29:18.514324 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:29:18.514324 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-df018152-dfbe-4c76-9255-f069d6be6ed4 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Returning 409 to user: Cannot 'detach_volume' instance a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 while it is in vm_state rescued {{(pid=118352) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:29:18.515461 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-df018152-dfbe-4c76-9255-f069d6be6ed4 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "DELETE /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24/os-volume_attachments/7cdab395-b182-49c3-8bf5-3e6673f0dc52" status: 409 len: 152 microversion: 2.1 time: 0.102632 Aug 08 02:29:18.515461 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 255/510] 10.222.0.27 () {58 vars in 1416 bytes} [Mon Aug 8 02:29:18 2022] DELETE /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24/os-volume_attachments/7cdab395-b182-49c3-8bf5-3e6673f0dc52 => generated 152 bytes in 104 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Aug 08 02:29:18.518465 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-39cda87f-9945-40fc-be49-93f8eae16f48 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:29:18.519167 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-39cda87f-9945-40fc-be49-93f8eae16f48 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:18.525469 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-39cda87f-9945-40fc-be49-93f8eae16f48 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:18.526115 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-39cda87f-9945-40fc-be49-93f8eae16f48 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:18.526396 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-39cda87f-9945-40fc-be49-93f8eae16f48 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:18.527097 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-39cda87f-9945-40fc-be49-93f8eae16f48 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:18.527555 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-39cda87f-9945-40fc-be49-93f8eae16f48 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:18.528048 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-39cda87f-9945-40fc-be49-93f8eae16f48 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:18.765574 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-39cda87f-9945-40fc-be49-93f8eae16f48 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "POST /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24/action" status: 202 len: 0 microversion: 2.1 time: 0.251482 Aug 08 02:29:18.766402 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 256/511] 10.222.0.27 () {60 vars in 1278 bytes} [Mon Aug 8 02:29:18 2022] POST /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24/action => generated 0 bytes in 253 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 08 02:29:18.776375 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-3172a50e-c72a-4b1d-b4e1-ffe02076772a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:18.777251 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-3172a50e-c72a-4b1d-b4e1-ffe02076772a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:18.786643 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3172a50e-c72a-4b1d-b4e1-ffe02076772a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:18.787134 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3172a50e-c72a-4b1d-b4e1-ffe02076772a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:18.787771 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3172a50e-c72a-4b1d-b4e1-ffe02076772a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:18.788711 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3172a50e-c72a-4b1d-b4e1-ffe02076772a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:18.789327 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3172a50e-c72a-4b1d-b4e1-ffe02076772a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:18.790018 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3172a50e-c72a-4b1d-b4e1-ffe02076772a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:18.942285 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-3172a50e-c72a-4b1d-b4e1-ffe02076772a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:18.951689 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-3172a50e-c72a-4b1d-b4e1-ffe02076772a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1724 microversion: 2.1 time: 0.180435 Aug 08 02:29:18.952263 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 256/512] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:18 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1724 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:19.973785 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-506aca26-be94-4104-a757-38e96389f21a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:19.974494 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-506aca26-be94-4104-a757-38e96389f21a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:19.981143 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-506aca26-be94-4104-a757-38e96389f21a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:19.981623 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-506aca26-be94-4104-a757-38e96389f21a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:19.982089 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-506aca26-be94-4104-a757-38e96389f21a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:19.982782 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-506aca26-be94-4104-a757-38e96389f21a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:19.984369 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-506aca26-be94-4104-a757-38e96389f21a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:19.984369 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-506aca26-be94-4104-a757-38e96389f21a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:20.127916 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-506aca26-be94-4104-a757-38e96389f21a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:20.135877 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-506aca26-be94-4104-a757-38e96389f21a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1724 microversion: 2.1 time: 0.166449 Aug 08 02:29:20.136436 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 257/513] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:19 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1724 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:20.658044 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-d8435fd9-6a23-42c3-9a8f-4009c67d243d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-348542991", "imageRef": "30833f47-5987-41ca-b5ee-ad5ecafbb265", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "f92c936a-6950-4530-8001-5260ea91adac"}], "security_groups": [{"name": "tempest-securitygroup--31162518"}], "key_name": "tempest-keypair-1247410672", "user_data": "}} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:29:20.760044 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.network.neutron [None req-d8435fd9-6a23-42c3-9a8f-4009c67d243d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] validate_networks() for [('f92c936a-6950-4530-8001-5260ea91adac', None, None, None, None, None)] {{(pid=118352) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 08 02:29:21.124409 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-d8435fd9-6a23-42c3-9a8f-4009c67d243d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Flavor limits 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 08 02:29:21.124911 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-d8435fd9-6a23-42c3-9a8f-4009c67d243d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Image limits 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 08 02:29:21.125393 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-d8435fd9-6a23-42c3-9a8f-4009c67d243d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Flavor pref 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 08 02:29:21.125835 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-d8435fd9-6a23-42c3-9a8f-4009c67d243d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Image pref 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 08 02:29:21.126412 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-d8435fd9-6a23-42c3-9a8f-4009c67d243d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 08 02:29:21.135210 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d8435fd9-6a23-42c3-9a8f-4009c67d243d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:21.135695 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d8435fd9-6a23-42c3-9a8f-4009c67d243d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:21.136133 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d8435fd9-6a23-42c3-9a8f-4009c67d243d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:21.136916 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d8435fd9-6a23-42c3-9a8f-4009c67d243d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:21.137375 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d8435fd9-6a23-42c3-9a8f-4009c67d243d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:21.137803 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d8435fd9-6a23-42c3-9a8f-4009c67d243d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:21.154790 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.quota [None req-d8435fd9-6a23-42c3-9a8f-4009c67d243d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Getting quotas for project ca29b2b9a59d4826bfd25f4dcfa9cffe. Resources: {'ram', 'cores', 'instances'} {{(pid=118352) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 08 02:29:21.158494 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-b4430e51-2109-4be8-9de6-e1179696ff99 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:21.159279 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-b4430e51-2109-4be8-9de6-e1179696ff99 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:21.159461 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.quota [None req-d8435fd9-6a23-42c3-9a8f-4009c67d243d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Getting quotas for user 578cd837d9fe442587f8c6c8528334c0 and project ca29b2b9a59d4826bfd25f4dcfa9cffe. Resources: {'ram', 'cores', 'instances'} {{(pid=118352) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 08 02:29:21.165843 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b4430e51-2109-4be8-9de6-e1179696ff99 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:21.166627 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b4430e51-2109-4be8-9de6-e1179696ff99 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:21.167068 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b4430e51-2109-4be8-9de6-e1179696ff99 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:21.167761 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b4430e51-2109-4be8-9de6-e1179696ff99 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:21.168220 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b4430e51-2109-4be8-9de6-e1179696ff99 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:21.168628 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b4430e51-2109-4be8-9de6-e1179696ff99 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:21.171424 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-d8435fd9-6a23-42c3-9a8f-4009c67d243d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Going to run 1 instances... {{(pid=118352) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Aug 08 02:29:21.173789 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-d8435fd9-6a23-42c3-9a8f-4009c67d243d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: aa20d25c-bd00-41e6-b350-309368839578] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='30833f47-5987-41ca-b5ee-ad5ecafbb265',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118352) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Aug 08 02:29:21.311631 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-b4430e51-2109-4be8-9de6-e1179696ff99 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:21.322030 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-b4430e51-2109-4be8-9de6-e1179696ff99 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1724 microversion: 2.1 time: 0.167836 Aug 08 02:29:21.323181 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 258/514] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:21 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1724 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:29:21.324157 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-d8435fd9-6a23-42c3-9a8f-4009c67d243d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 400 microversion: 2.1 time: 0.671064 Aug 08 02:29:21.324986 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 257/515] 10.222.0.27 () {60 vars in 1147 bytes} [Mon Aug 8 02:29:20 2022] POST /compute/v2.1/servers => generated 400 bytes in 672 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 08 02:29:21.336584 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-ca9013da-1d92-4c3a-af7d-02ae1198941e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:21.340303 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-ca9013da-1d92-4c3a-af7d-02ae1198941e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: aa20d25c-bd00-41e6-b350-309368839578] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:21.390565 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-ca9013da-1d92-4c3a-af7d-02ae1198941e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:21.400087 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-ca9013da-1d92-4c3a-af7d-02ae1198941e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578" status: 200 len: 1241 microversion: 2.1 time: 0.067447 Aug 08 02:29:21.400659 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 259/516] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:21 2022] GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578 => generated 1241 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:29:22.344318 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-b4586f5f-6b0f-401e-90ba-d073b4bb258d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:22.345189 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-b4586f5f-6b0f-401e-90ba-d073b4bb258d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:22.351903 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b4586f5f-6b0f-401e-90ba-d073b4bb258d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:22.352393 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b4586f5f-6b0f-401e-90ba-d073b4bb258d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:22.352868 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b4586f5f-6b0f-401e-90ba-d073b4bb258d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:22.353563 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b4586f5f-6b0f-401e-90ba-d073b4bb258d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:22.354153 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b4586f5f-6b0f-401e-90ba-d073b4bb258d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:22.354638 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b4586f5f-6b0f-401e-90ba-d073b4bb258d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:22.420747 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-6d5c1d1c-1d92-468b-9f2c-f35983ec2551 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:22.421360 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-6d5c1d1c-1d92-468b-9f2c-f35983ec2551 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: aa20d25c-bd00-41e6-b350-309368839578] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:22.427908 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6d5c1d1c-1d92-468b-9f2c-f35983ec2551 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:22.428579 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6d5c1d1c-1d92-468b-9f2c-f35983ec2551 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:22.429206 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6d5c1d1c-1d92-468b-9f2c-f35983ec2551 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:22.430074 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6d5c1d1c-1d92-468b-9f2c-f35983ec2551 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:22.430527 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6d5c1d1c-1d92-468b-9f2c-f35983ec2551 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:22.430967 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6d5c1d1c-1d92-468b-9f2c-f35983ec2551 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:22.508724 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-6d5c1d1c-1d92-468b-9f2c-f35983ec2551 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:22.517764 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-6d5c1d1c-1d92-468b-9f2c-f35983ec2551 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578" status: 200 len: 1245 microversion: 2.1 time: 0.099249 Aug 08 02:29:22.517764 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 260/517] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:22 2022] GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578 => generated 1245 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:29:22.524019 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-b4586f5f-6b0f-401e-90ba-d073b4bb258d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:22.537056 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-b4586f5f-6b0f-401e-90ba-d073b4bb258d tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1724 microversion: 2.1 time: 0.197232 Aug 08 02:29:22.537625 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 258/518] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:22 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1724 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:23.539124 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-74cca946-17dd-4f89-a7e2-9d8f04900d1e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:23.539733 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-74cca946-17dd-4f89-a7e2-9d8f04900d1e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: aa20d25c-bd00-41e6-b350-309368839578] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:23.546125 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-74cca946-17dd-4f89-a7e2-9d8f04900d1e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:23.546610 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-74cca946-17dd-4f89-a7e2-9d8f04900d1e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:23.547029 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-74cca946-17dd-4f89-a7e2-9d8f04900d1e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:23.547706 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-74cca946-17dd-4f89-a7e2-9d8f04900d1e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:23.548157 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-74cca946-17dd-4f89-a7e2-9d8f04900d1e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:23.548573 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-74cca946-17dd-4f89-a7e2-9d8f04900d1e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:23.561302 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-43b9a513-7e05-4e67-90fe-d66fd11cc011 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:23.561302 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-43b9a513-7e05-4e67-90fe-d66fd11cc011 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:23.567771 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-43b9a513-7e05-4e67-90fe-d66fd11cc011 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:23.568045 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-43b9a513-7e05-4e67-90fe-d66fd11cc011 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:23.568558 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-43b9a513-7e05-4e67-90fe-d66fd11cc011 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:23.569342 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-43b9a513-7e05-4e67-90fe-d66fd11cc011 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:23.569928 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-43b9a513-7e05-4e67-90fe-d66fd11cc011 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:23.570423 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-43b9a513-7e05-4e67-90fe-d66fd11cc011 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:23.637039 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-74cca946-17dd-4f89-a7e2-9d8f04900d1e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:23.646681 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-74cca946-17dd-4f89-a7e2-9d8f04900d1e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578" status: 200 len: 1293 microversion: 2.1 time: 0.111912 Aug 08 02:29:23.647259 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 261/519] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:23 2022] GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578 => generated 1293 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:23.722861 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-43b9a513-7e05-4e67-90fe-d66fd11cc011 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:23.731045 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-43b9a513-7e05-4e67-90fe-d66fd11cc011 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1730 microversion: 2.1 time: 0.175639 Aug 08 02:29:23.731607 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 259/520] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:23 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1730 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:23.731745 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-eafc0f13-c7ed-4b32-a8b8-a4d6fc518c6e tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestShelveInstance-server-1031665227", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestShelveInstance-556747333", "security_groups": [{"name": "tempest-secgroup-smoke-813371614"}], "networks": [{"uuid": "6fe4a268-f47a-420a-80c5-dd33e7e621f8"}], "block_device_mapping_v2": [{"uuid": "19b60398-f50b-43ce-95ee-cc3bbc87b5b4", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:29:23.751058 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-09de4932-31dd-447e-9a17-c030a83b8d13 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:23.754450 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-09de4932-31dd-447e-9a17-c030a83b8d13 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:23.760576 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-09de4932-31dd-447e-9a17-c030a83b8d13 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:23.761095 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-09de4932-31dd-447e-9a17-c030a83b8d13 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:23.761959 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-09de4932-31dd-447e-9a17-c030a83b8d13 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:23.762535 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-09de4932-31dd-447e-9a17-c030a83b8d13 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:23.762997 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-09de4932-31dd-447e-9a17-c030a83b8d13 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:23.763548 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-09de4932-31dd-447e-9a17-c030a83b8d13 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:23.858012 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-09de4932-31dd-447e-9a17-c030a83b8d13 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.110590 Aug 08 02:29:23.858636 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 260/521] 10.222.0.27 () {58 vars in 1275 bytes} [Mon Aug 8 02:29:23 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24/os-interface => generated 284 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 08 02:29:23.919967 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.network.neutron [None req-eafc0f13-c7ed-4b32-a8b8-a4d6fc518c6e tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] validate_networks() for [('6fe4a268-f47a-420a-80c5-dd33e7e621f8', None, None, None, None, None)] {{(pid=118351) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 08 02:29:24.670012 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-414fdcc5-f7a5-4e8f-86ea-e97fca4bd069 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:24.671788 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-414fdcc5-f7a5-4e8f-86ea-e97fca4bd069 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: aa20d25c-bd00-41e6-b350-309368839578] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:24.680101 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-414fdcc5-f7a5-4e8f-86ea-e97fca4bd069 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:24.680805 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-414fdcc5-f7a5-4e8f-86ea-e97fca4bd069 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:24.682594 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-414fdcc5-f7a5-4e8f-86ea-e97fca4bd069 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:24.683833 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-414fdcc5-f7a5-4e8f-86ea-e97fca4bd069 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:24.684431 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-414fdcc5-f7a5-4e8f-86ea-e97fca4bd069 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:24.684988 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-414fdcc5-f7a5-4e8f-86ea-e97fca4bd069 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:24.900412 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.virt.hardware [None req-eafc0f13-c7ed-4b32-a8b8-a4d6fc518c6e tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Flavor limits 0:0:0 {{(pid=118351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 08 02:29:24.900921 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.virt.hardware [None req-eafc0f13-c7ed-4b32-a8b8-a4d6fc518c6e tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Image limits 0:0:0 {{(pid=118351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 08 02:29:24.901334 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.virt.hardware [None req-eafc0f13-c7ed-4b32-a8b8-a4d6fc518c6e tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Flavor pref 0:0:0 {{(pid=118351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 08 02:29:24.901743 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.virt.hardware [None req-eafc0f13-c7ed-4b32-a8b8-a4d6fc518c6e tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Image pref 0:0:0 {{(pid=118351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 08 02:29:24.902209 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.virt.hardware [None req-eafc0f13-c7ed-4b32-a8b8-a4d6fc518c6e tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 08 02:29:24.911491 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-eafc0f13-c7ed-4b32-a8b8-a4d6fc518c6e tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:24.911961 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-eafc0f13-c7ed-4b32-a8b8-a4d6fc518c6e tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:24.912392 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-eafc0f13-c7ed-4b32-a8b8-a4d6fc518c6e tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:24.913189 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-eafc0f13-c7ed-4b32-a8b8-a4d6fc518c6e tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:24.913643 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-eafc0f13-c7ed-4b32-a8b8-a4d6fc518c6e tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:24.914066 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-eafc0f13-c7ed-4b32-a8b8-a4d6fc518c6e tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:24.919868 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-414fdcc5-f7a5-4e8f-86ea-e97fca4bd069 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:24.932568 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.quota [None req-eafc0f13-c7ed-4b32-a8b8-a4d6fc518c6e tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Getting quotas for project fa4edbfca7b749ff9be3268fa6b81264. Resources: {'ram', 'cores', 'instances'} {{(pid=118351) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 08 02:29:24.936705 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.quota [None req-eafc0f13-c7ed-4b32-a8b8-a4d6fc518c6e tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Getting quotas for user 8be4cb4f1c594f42825eb40985a29cac and project fa4edbfca7b749ff9be3268fa6b81264. Resources: {'ram', 'cores', 'instances'} {{(pid=118351) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 08 02:29:24.945743 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-414fdcc5-f7a5-4e8f-86ea-e97fca4bd069 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578" status: 200 len: 1301 microversion: 2.1 time: 0.281310 Aug 08 02:29:24.949067 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 261/522] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:24 2022] GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578 => generated 1301 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:29:25.030162 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-eafc0f13-c7ed-4b32-a8b8-a4d6fc518c6e tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Going to run 1 instances... {{(pid=118351) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Aug 08 02:29:25.032071 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-eafc0f13-c7ed-4b32-a8b8-a4d6fc518c6e tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='19b60398-f50b-43ce-95ee-cc3bbc87b5b4',volume_size=None,volume_type=None)] {{(pid=118351) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Aug 08 02:29:25.428844 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [req-e612fa9c-80f8-4620-89ba-19b799bc3c72 req-1f135806-57cd-40fd-b1b9-ceb6b4615db3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a5b1155c-5bba-46ae-8f3c-61f70c8a8a24", "tag": "9b3bb00c-894f-4b64-935c-89a64519fd90"}]} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:29:25.438530 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-e612fa9c-80f8-4620-89ba-19b799bc3c72 req-1f135806-57cd-40fd-b1b9-ceb6b4615db3 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:25.439253 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-e612fa9c-80f8-4620-89ba-19b799bc3c72 req-1f135806-57cd-40fd-b1b9-ceb6b4615db3 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:25.439683 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-e612fa9c-80f8-4620-89ba-19b799bc3c72 req-1f135806-57cd-40fd-b1b9-ceb6b4615db3 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:25.451876 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.compute.server_external_events [req-e612fa9c-80f8-4620-89ba-19b799bc3c72 req-1f135806-57cd-40fd-b1b9-ceb6b4615db3 service nova] Creating event network-changed:9b3bb00c-894f-4b64-935c-89a64519fd90 for instance a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 on hv-cinder-74014 Aug 08 02:29:25.472338 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [req-e612fa9c-80f8-4620-89ba-19b799bc3c72 req-1f135806-57cd-40fd-b1b9-ceb6b4615db3 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.048340 Aug 08 02:29:25.474228 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 262/523] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:29:25 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 08 02:29:25.656697 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-eafc0f13-c7ed-4b32-a8b8-a4d6fc518c6e tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 1.928726 Aug 08 02:29:25.669729 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-eaf6bead-9537-4855-b928-b7eb17696033 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:25.670769 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-eaf6bead-9537-4855-b928-b7eb17696033 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:25.675028 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 262/524] 10.222.0.27 () {60 vars in 1147 bytes} [Mon Aug 8 02:29:23 2022] POST /compute/v2.1/servers => generated 405 bytes in 1947 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Aug 08 02:29:25.743965 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-eaf6bead-9537-4855-b928-b7eb17696033 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:25.756082 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-eaf6bead-9537-4855-b928-b7eb17696033 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1088 microversion: 2.1 time: 0.090068 Aug 08 02:29:25.757195 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 263/525] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:25 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1088 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:29:25.968530 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-b9078135-29ec-47a3-9515-d5377bcab198 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:25.969243 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-b9078135-29ec-47a3-9515-d5377bcab198 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: aa20d25c-bd00-41e6-b350-309368839578] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:25.984063 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b9078135-29ec-47a3-9515-d5377bcab198 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:25.984945 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b9078135-29ec-47a3-9515-d5377bcab198 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:25.985453 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b9078135-29ec-47a3-9515-d5377bcab198 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:25.986250 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b9078135-29ec-47a3-9515-d5377bcab198 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:25.987037 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b9078135-29ec-47a3-9515-d5377bcab198 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:25.987572 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b9078135-29ec-47a3-9515-d5377bcab198 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:26.138655 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-b9078135-29ec-47a3-9515-d5377bcab198 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:26.146316 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-b9078135-29ec-47a3-9515-d5377bcab198 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578" status: 200 len: 1365 microversion: 2.1 time: 0.182759 Aug 08 02:29:26.146885 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 263/526] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:25 2022] GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578 => generated 1365 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:26.776984 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-acc9166e-cc12-4b98-ae55-595658ff2e24 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:26.777634 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-acc9166e-cc12-4b98-ae55-595658ff2e24 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:26.830435 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-acc9166e-cc12-4b98-ae55-595658ff2e24 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:26.837770 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-acc9166e-cc12-4b98-ae55-595658ff2e24 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1088 microversion: 2.1 time: 0.064792 Aug 08 02:29:26.838336 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 264/527] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:26 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1088 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:27.166759 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-b86d8430-dfdc-4567-93c7-2592c2b034a9 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:27.167402 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-b86d8430-dfdc-4567-93c7-2592c2b034a9 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: aa20d25c-bd00-41e6-b350-309368839578] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:27.173894 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b86d8430-dfdc-4567-93c7-2592c2b034a9 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:27.176424 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b86d8430-dfdc-4567-93c7-2592c2b034a9 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:27.176424 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b86d8430-dfdc-4567-93c7-2592c2b034a9 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:27.176424 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b86d8430-dfdc-4567-93c7-2592c2b034a9 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:27.176754 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b86d8430-dfdc-4567-93c7-2592c2b034a9 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:27.177154 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b86d8430-dfdc-4567-93c7-2592c2b034a9 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:27.307068 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-b86d8430-dfdc-4567-93c7-2592c2b034a9 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:27.314606 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-b86d8430-dfdc-4567-93c7-2592c2b034a9 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578" status: 200 len: 1365 microversion: 2.1 time: 0.151747 Aug 08 02:29:27.315162 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 264/528] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:27 2022] GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578 => generated 1365 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:27.490149 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [req-22f32804-72f6-4fa9-af6d-3a3d9e8588b6 req-e40e2d0e-e12c-461a-9de7-fdaf9b89f9d1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a5b1155c-5bba-46ae-8f3c-61f70c8a8a24", "tag": "9b3bb00c-894f-4b64-935c-89a64519fd90"}]} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:29:27.499596 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-22f32804-72f6-4fa9-af6d-3a3d9e8588b6 req-e40e2d0e-e12c-461a-9de7-fdaf9b89f9d1 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:27.500072 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-22f32804-72f6-4fa9-af6d-3a3d9e8588b6 req-e40e2d0e-e12c-461a-9de7-fdaf9b89f9d1 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:27.500507 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-22f32804-72f6-4fa9-af6d-3a3d9e8588b6 req-e40e2d0e-e12c-461a-9de7-fdaf9b89f9d1 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:27.512746 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.compute.server_external_events [req-22f32804-72f6-4fa9-af6d-3a3d9e8588b6 req-e40e2d0e-e12c-461a-9de7-fdaf9b89f9d1 service nova] Creating event network-changed:9b3bb00c-894f-4b64-935c-89a64519fd90 for instance a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 on hv-cinder-74014 Aug 08 02:29:27.517495 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [req-22f32804-72f6-4fa9-af6d-3a3d9e8588b6 req-e40e2d0e-e12c-461a-9de7-fdaf9b89f9d1 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031717 Aug 08 02:29:27.518060 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 265/529] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:29:27 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:29:27.857569 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-508ff59a-bdc7-412a-8d2b-19da258c6646 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:27.858210 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-508ff59a-bdc7-412a-8d2b-19da258c6646 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:27.864391 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-508ff59a-bdc7-412a-8d2b-19da258c6646 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:27.864996 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-508ff59a-bdc7-412a-8d2b-19da258c6646 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:27.865435 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-508ff59a-bdc7-412a-8d2b-19da258c6646 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:27.866131 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-508ff59a-bdc7-412a-8d2b-19da258c6646 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:27.866589 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-508ff59a-bdc7-412a-8d2b-19da258c6646 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:27.867014 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-508ff59a-bdc7-412a-8d2b-19da258c6646 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:27.927500 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-508ff59a-bdc7-412a-8d2b-19da258c6646 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:27.935080 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-508ff59a-bdc7-412a-8d2b-19da258c6646 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1138 microversion: 2.1 time: 0.081323 Aug 08 02:29:27.935642 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 265/530] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:27 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1138 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:28.336551 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-116f6666-33c4-4681-9337-bf07c1acaca7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:28.337959 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-116f6666-33c4-4681-9337-bf07c1acaca7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: aa20d25c-bd00-41e6-b350-309368839578] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:28.355173 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-116f6666-33c4-4681-9337-bf07c1acaca7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:28.355652 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-116f6666-33c4-4681-9337-bf07c1acaca7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:28.356340 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-116f6666-33c4-4681-9337-bf07c1acaca7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:28.357074 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-116f6666-33c4-4681-9337-bf07c1acaca7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:28.357532 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-116f6666-33c4-4681-9337-bf07c1acaca7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:28.357961 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-116f6666-33c4-4681-9337-bf07c1acaca7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:28.500587 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-116f6666-33c4-4681-9337-bf07c1acaca7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:28.509056 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-116f6666-33c4-4681-9337-bf07c1acaca7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578" status: 200 len: 1365 microversion: 2.1 time: 0.177329 Aug 08 02:29:28.514195 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 266/531] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:28 2022] GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578 => generated 1365 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:29:28.955571 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-7a47fedc-767d-4bb1-92bf-8b0a90f257c5 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:28.956175 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-7a47fedc-767d-4bb1-92bf-8b0a90f257c5 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:28.963841 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7a47fedc-767d-4bb1-92bf-8b0a90f257c5 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:28.964352 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7a47fedc-767d-4bb1-92bf-8b0a90f257c5 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:28.964948 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7a47fedc-767d-4bb1-92bf-8b0a90f257c5 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:28.965745 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7a47fedc-767d-4bb1-92bf-8b0a90f257c5 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:28.966233 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7a47fedc-767d-4bb1-92bf-8b0a90f257c5 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:28.966654 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7a47fedc-767d-4bb1-92bf-8b0a90f257c5 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:29.029982 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-7a47fedc-767d-4bb1-92bf-8b0a90f257c5 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:29.037479 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-7a47fedc-767d-4bb1-92bf-8b0a90f257c5 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1186 microversion: 2.1 time: 0.085988 Aug 08 02:29:29.038045 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 266/532] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:28 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1186 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:29.388056 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [req-3bf65c36-6159-4ac8-885e-af58f2ed5473 req-9df8b1d2-ae23-4f0a-a7b3-fceea7de8db7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "aa20d25c-bd00-41e6-b350-309368839578", "tag": "72aebc3e-9c8c-4a19-8c44-a4ea8d03d79e"}]} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:29:29.406484 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-3bf65c36-6159-4ac8-885e-af58f2ed5473 req-9df8b1d2-ae23-4f0a-a7b3-fceea7de8db7 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:29.407115 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-3bf65c36-6159-4ac8-885e-af58f2ed5473 req-9df8b1d2-ae23-4f0a-a7b3-fceea7de8db7 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:29.407644 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-3bf65c36-6159-4ac8-885e-af58f2ed5473 req-9df8b1d2-ae23-4f0a-a7b3-fceea7de8db7 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:29.419567 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.compute.server_external_events [req-3bf65c36-6159-4ac8-885e-af58f2ed5473 req-9df8b1d2-ae23-4f0a-a7b3-fceea7de8db7 service nova] Creating event network-changed:72aebc3e-9c8c-4a19-8c44-a4ea8d03d79e for instance aa20d25c-bd00-41e6-b350-309368839578 on hv-cinder-74014 Aug 08 02:29:29.424550 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [req-3bf65c36-6159-4ac8-885e-af58f2ed5473 req-9df8b1d2-ae23-4f0a-a7b3-fceea7de8db7 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.040943 Aug 08 02:29:29.425290 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 267/533] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:29:29 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:29:29.529865 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-1f44ead7-4935-4f50-9df7-51e2dff6c6c5 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:29.530472 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-1f44ead7-4935-4f50-9df7-51e2dff6c6c5 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: aa20d25c-bd00-41e6-b350-309368839578] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:29.537717 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1f44ead7-4935-4f50-9df7-51e2dff6c6c5 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:29.538205 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1f44ead7-4935-4f50-9df7-51e2dff6c6c5 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:29.538638 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1f44ead7-4935-4f50-9df7-51e2dff6c6c5 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:29.539324 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1f44ead7-4935-4f50-9df7-51e2dff6c6c5 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:29.540201 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1f44ead7-4935-4f50-9df7-51e2dff6c6c5 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:29.540803 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1f44ead7-4935-4f50-9df7-51e2dff6c6c5 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:29.685991 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-1f44ead7-4935-4f50-9df7-51e2dff6c6c5 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:29.694385 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-1f44ead7-4935-4f50-9df7-51e2dff6c6c5 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578" status: 200 len: 1365 microversion: 2.1 time: 0.168298 Aug 08 02:29:29.695285 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 267/534] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:29 2022] GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578 => generated 1365 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:30.058858 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-ea63520e-cc1c-42de-96f2-bdb99ef227e6 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:30.063699 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-ea63520e-cc1c-42de-96f2-bdb99ef227e6 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:30.066517 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ea63520e-cc1c-42de-96f2-bdb99ef227e6 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:30.067023 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ea63520e-cc1c-42de-96f2-bdb99ef227e6 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:30.067746 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ea63520e-cc1c-42de-96f2-bdb99ef227e6 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:30.068506 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ea63520e-cc1c-42de-96f2-bdb99ef227e6 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:30.069164 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ea63520e-cc1c-42de-96f2-bdb99ef227e6 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:30.069744 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ea63520e-cc1c-42de-96f2-bdb99ef227e6 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:30.187551 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-ea63520e-cc1c-42de-96f2-bdb99ef227e6 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:30.196887 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-ea63520e-cc1c-42de-96f2-bdb99ef227e6 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1204 microversion: 2.1 time: 0.142311 Aug 08 02:29:30.197472 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 268/535] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:30 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1204 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:30.714972 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-4a9607e1-623b-42bc-b643-486f5d96a1db tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:30.715589 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-4a9607e1-623b-42bc-b643-486f5d96a1db tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: aa20d25c-bd00-41e6-b350-309368839578] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:30.722092 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4a9607e1-623b-42bc-b643-486f5d96a1db tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:30.722579 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4a9607e1-623b-42bc-b643-486f5d96a1db tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:30.723275 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4a9607e1-623b-42bc-b643-486f5d96a1db tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:30.723912 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4a9607e1-623b-42bc-b643-486f5d96a1db tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:30.724371 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4a9607e1-623b-42bc-b643-486f5d96a1db tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:30.724826 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4a9607e1-623b-42bc-b643-486f5d96a1db tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:30.859602 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-4a9607e1-623b-42bc-b643-486f5d96a1db tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:30.867116 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-4a9607e1-623b-42bc-b643-486f5d96a1db tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578" status: 200 len: 1365 microversion: 2.1 time: 0.156095 Aug 08 02:29:30.867671 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 268/536] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:30 2022] GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578 => generated 1365 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:31.219370 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-6aa56331-b794-4664-b2c5-83773212e52c tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:31.220179 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-6aa56331-b794-4664-b2c5-83773212e52c tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:31.222669 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell0:SELECT=2 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:29:31.228291 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6aa56331-b794-4664-b2c5-83773212e52c tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:31.228901 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6aa56331-b794-4664-b2c5-83773212e52c tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:31.229337 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6aa56331-b794-4664-b2c5-83773212e52c tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:31.230026 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6aa56331-b794-4664-b2c5-83773212e52c tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:31.230474 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6aa56331-b794-4664-b2c5-83773212e52c tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:31.230896 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6aa56331-b794-4664-b2c5-83773212e52c tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:31.359834 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-6aa56331-b794-4664-b2c5-83773212e52c tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:31.367218 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-6aa56331-b794-4664-b2c5-83773212e52c tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1271 microversion: 2.1 time: 0.151886 Aug 08 02:29:31.367777 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 269/537] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:31 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1271 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:31.887713 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-1ab3c784-8591-4e18-9790-a04b94464efa tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:31.888325 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-1ab3c784-8591-4e18-9790-a04b94464efa tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: aa20d25c-bd00-41e6-b350-309368839578] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:31.895141 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1ab3c784-8591-4e18-9790-a04b94464efa tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:31.895621 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1ab3c784-8591-4e18-9790-a04b94464efa tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:31.896050 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1ab3c784-8591-4e18-9790-a04b94464efa tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:31.896748 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1ab3c784-8591-4e18-9790-a04b94464efa tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:31.897194 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1ab3c784-8591-4e18-9790-a04b94464efa tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:31.897610 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1ab3c784-8591-4e18-9790-a04b94464efa tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:32.027233 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-1ab3c784-8591-4e18-9790-a04b94464efa tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:32.030743 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:29:32.034923 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-1ab3c784-8591-4e18-9790-a04b94464efa tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578" status: 200 len: 1365 microversion: 2.1 time: 0.151025 Aug 08 02:29:32.041045 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 269/538] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:31 2022] GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578 => generated 1365 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:32.387350 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-4bac5917-f9f5-4a96-963d-7780a17088b1 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:32.387966 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-4bac5917-f9f5-4a96-963d-7780a17088b1 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:32.395092 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4bac5917-f9f5-4a96-963d-7780a17088b1 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:32.395578 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4bac5917-f9f5-4a96-963d-7780a17088b1 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:32.396018 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4bac5917-f9f5-4a96-963d-7780a17088b1 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:32.396725 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4bac5917-f9f5-4a96-963d-7780a17088b1 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:32.397183 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4bac5917-f9f5-4a96-963d-7780a17088b1 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:32.397610 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4bac5917-f9f5-4a96-963d-7780a17088b1 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:32.531786 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-4bac5917-f9f5-4a96-963d-7780a17088b1 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:32.540113 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-4bac5917-f9f5-4a96-963d-7780a17088b1 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1271 microversion: 2.1 time: 0.156693 Aug 08 02:29:32.540685 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 270/539] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:32 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1271 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:33.057123 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-60fc93d6-3e2d-49ac-bcae-beb5a87602fb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:33.057739 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-60fc93d6-3e2d-49ac-bcae-beb5a87602fb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: aa20d25c-bd00-41e6-b350-309368839578] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:33.064807 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-60fc93d6-3e2d-49ac-bcae-beb5a87602fb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:33.065287 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-60fc93d6-3e2d-49ac-bcae-beb5a87602fb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:33.065723 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-60fc93d6-3e2d-49ac-bcae-beb5a87602fb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:33.067142 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-60fc93d6-3e2d-49ac-bcae-beb5a87602fb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:33.067888 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-60fc93d6-3e2d-49ac-bcae-beb5a87602fb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:33.068317 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-60fc93d6-3e2d-49ac-bcae-beb5a87602fb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:33.197207 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-60fc93d6-3e2d-49ac-bcae-beb5a87602fb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:33.204633 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-60fc93d6-3e2d-49ac-bcae-beb5a87602fb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578" status: 200 len: 1365 microversion: 2.1 time: 0.151416 Aug 08 02:29:33.205223 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 270/540] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:33 2022] GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578 => generated 1365 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:33.562525 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-7068a886-1c2e-4aaf-b0fe-43ec60025323 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:33.563127 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-7068a886-1c2e-4aaf-b0fe-43ec60025323 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:33.571229 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7068a886-1c2e-4aaf-b0fe-43ec60025323 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:33.571643 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7068a886-1c2e-4aaf-b0fe-43ec60025323 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:33.572073 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7068a886-1c2e-4aaf-b0fe-43ec60025323 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:33.572786 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7068a886-1c2e-4aaf-b0fe-43ec60025323 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:33.573240 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7068a886-1c2e-4aaf-b0fe-43ec60025323 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:33.576762 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7068a886-1c2e-4aaf-b0fe-43ec60025323 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:33.715449 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-7068a886-1c2e-4aaf-b0fe-43ec60025323 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:33.723117 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-7068a886-1c2e-4aaf-b0fe-43ec60025323 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1271 microversion: 2.1 time: 0.164462 Aug 08 02:29:33.723679 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 271/541] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:33 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1271 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:34.020192 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [req-38c49f8b-3387-4e20-8564-09e6751cf80b req-e4bd23db-69db-48c2-8c5d-2f9f12590a31 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "077e7935-3361-4ff5-9367-ba162efac09c", "tag": "3051cabc-48a3-4055-ae74-0ae74aa73005"}]} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:29:34.029927 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-38c49f8b-3387-4e20-8564-09e6751cf80b req-e4bd23db-69db-48c2-8c5d-2f9f12590a31 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:34.030432 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-38c49f8b-3387-4e20-8564-09e6751cf80b req-e4bd23db-69db-48c2-8c5d-2f9f12590a31 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:34.030860 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-38c49f8b-3387-4e20-8564-09e6751cf80b req-e4bd23db-69db-48c2-8c5d-2f9f12590a31 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:34.041684 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.compute.server_external_events [req-38c49f8b-3387-4e20-8564-09e6751cf80b req-e4bd23db-69db-48c2-8c5d-2f9f12590a31 service nova] Creating event network-changed:3051cabc-48a3-4055-ae74-0ae74aa73005 for instance 077e7935-3361-4ff5-9367-ba162efac09c on hv-cinder-74014 Aug 08 02:29:34.046573 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [req-38c49f8b-3387-4e20-8564-09e6751cf80b req-e4bd23db-69db-48c2-8c5d-2f9f12590a31 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030156 Aug 08 02:29:34.047161 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 271/542] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:29:34 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:29:34.226261 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-efac0520-ca82-4f2c-aefd-96997f21dfcb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:34.226797 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-efac0520-ca82-4f2c-aefd-96997f21dfcb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: aa20d25c-bd00-41e6-b350-309368839578] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:34.232792 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-efac0520-ca82-4f2c-aefd-96997f21dfcb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:34.233287 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-efac0520-ca82-4f2c-aefd-96997f21dfcb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:34.233727 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-efac0520-ca82-4f2c-aefd-96997f21dfcb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:34.234435 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-efac0520-ca82-4f2c-aefd-96997f21dfcb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:34.234878 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-efac0520-ca82-4f2c-aefd-96997f21dfcb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:34.235307 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-efac0520-ca82-4f2c-aefd-96997f21dfcb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:34.381343 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-efac0520-ca82-4f2c-aefd-96997f21dfcb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:34.389308 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-efac0520-ca82-4f2c-aefd-96997f21dfcb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578" status: 200 len: 1365 microversion: 2.1 time: 0.167621 Aug 08 02:29:34.389878 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 272/543] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:34 2022] GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578 => generated 1365 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:34.744330 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-c1b4c792-82ac-4f75-8ea7-317aa633821b tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:34.745152 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-c1b4c792-82ac-4f75-8ea7-317aa633821b tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:34.751490 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c1b4c792-82ac-4f75-8ea7-317aa633821b tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:34.751976 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c1b4c792-82ac-4f75-8ea7-317aa633821b tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:34.752407 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c1b4c792-82ac-4f75-8ea7-317aa633821b tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:34.753142 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c1b4c792-82ac-4f75-8ea7-317aa633821b tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:34.753601 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c1b4c792-82ac-4f75-8ea7-317aa633821b tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:34.754026 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c1b4c792-82ac-4f75-8ea7-317aa633821b tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:34.893068 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-c1b4c792-82ac-4f75-8ea7-317aa633821b tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:34.901812 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-c1b4c792-82ac-4f75-8ea7-317aa633821b tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1271 microversion: 2.1 time: 0.162154 Aug 08 02:29:34.902422 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 272/544] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:34 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1271 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:35.410622 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-cd627996-a174-4d06-8b52-cc5e11cbf125 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:35.411151 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-cd627996-a174-4d06-8b52-cc5e11cbf125 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: aa20d25c-bd00-41e6-b350-309368839578] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:35.417679 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cd627996-a174-4d06-8b52-cc5e11cbf125 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:35.418157 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cd627996-a174-4d06-8b52-cc5e11cbf125 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:35.418635 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cd627996-a174-4d06-8b52-cc5e11cbf125 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:35.419332 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cd627996-a174-4d06-8b52-cc5e11cbf125 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:35.419776 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cd627996-a174-4d06-8b52-cc5e11cbf125 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:35.420235 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cd627996-a174-4d06-8b52-cc5e11cbf125 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:35.551865 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-cd627996-a174-4d06-8b52-cc5e11cbf125 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:35.559398 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-cd627996-a174-4d06-8b52-cc5e11cbf125 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578" status: 200 len: 1365 microversion: 2.1 time: 0.152832 Aug 08 02:29:35.559961 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 273/545] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:35 2022] GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578 => generated 1365 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:35.922677 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-1b06ceb3-4985-43a4-b684-79074b5dd0c6 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:35.923315 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-1b06ceb3-4985-43a4-b684-79074b5dd0c6 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:35.925287 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell0:SELECT=2 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:29:35.927153 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell0:SELECT=8 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:29:35.932326 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1b06ceb3-4985-43a4-b684-79074b5dd0c6 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:35.943543 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1b06ceb3-4985-43a4-b684-79074b5dd0c6 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:35.943543 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1b06ceb3-4985-43a4-b684-79074b5dd0c6 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:35.943543 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1b06ceb3-4985-43a4-b684-79074b5dd0c6 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:35.943543 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1b06ceb3-4985-43a4-b684-79074b5dd0c6 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:35.943932 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1b06ceb3-4985-43a4-b684-79074b5dd0c6 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:36.066381 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-1b06ceb3-4985-43a4-b684-79074b5dd0c6 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:36.074191 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-1b06ceb3-4985-43a4-b684-79074b5dd0c6 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1259 microversion: 2.1 time: 0.155362 Aug 08 02:29:36.074757 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 273/546] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:35 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1259 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:36.587974 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-58a8021b-0753-4063-8067-241dd3bd6387 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:36.589026 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-58a8021b-0753-4063-8067-241dd3bd6387 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: aa20d25c-bd00-41e6-b350-309368839578] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:36.595255 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-58a8021b-0753-4063-8067-241dd3bd6387 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:36.595732 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-58a8021b-0753-4063-8067-241dd3bd6387 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:36.596165 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-58a8021b-0753-4063-8067-241dd3bd6387 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:36.596925 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-58a8021b-0753-4063-8067-241dd3bd6387 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:36.597376 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-58a8021b-0753-4063-8067-241dd3bd6387 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:36.598131 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-58a8021b-0753-4063-8067-241dd3bd6387 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:36.616454 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell1:SELECT=100 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:29:36.730325 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-58a8021b-0753-4063-8067-241dd3bd6387 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:36.738026 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-58a8021b-0753-4063-8067-241dd3bd6387 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578" status: 200 len: 1365 microversion: 2.1 time: 0.153935 Aug 08 02:29:36.739376 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 274/547] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:36 2022] GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578 => generated 1365 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:29:37.095788 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-2c52d276-9724-4d59-9cc9-315d345400a5 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:37.096399 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-2c52d276-9724-4d59-9cc9-315d345400a5 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:37.103289 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2c52d276-9724-4d59-9cc9-315d345400a5 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:37.103772 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2c52d276-9724-4d59-9cc9-315d345400a5 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:37.104209 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2c52d276-9724-4d59-9cc9-315d345400a5 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:37.104922 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2c52d276-9724-4d59-9cc9-315d345400a5 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:37.105376 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2c52d276-9724-4d59-9cc9-315d345400a5 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:37.105802 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2c52d276-9724-4d59-9cc9-315d345400a5 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:37.236709 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-2c52d276-9724-4d59-9cc9-315d345400a5 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:37.244762 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-2c52d276-9724-4d59-9cc9-315d345400a5 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1259 microversion: 2.1 time: 0.152822 Aug 08 02:29:37.245332 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 274/548] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:37 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1259 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:29:37.759978 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-1a6f6e44-40e6-49f9-b357-e59d6e981799 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:37.760643 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-1a6f6e44-40e6-49f9-b357-e59d6e981799 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: aa20d25c-bd00-41e6-b350-309368839578] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:37.762527 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell0:SELECT=14 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:29:37.769385 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1a6f6e44-40e6-49f9-b357-e59d6e981799 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:37.769492 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1a6f6e44-40e6-49f9-b357-e59d6e981799 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:37.769919 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1a6f6e44-40e6-49f9-b357-e59d6e981799 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:37.771033 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1a6f6e44-40e6-49f9-b357-e59d6e981799 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:37.771574 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1a6f6e44-40e6-49f9-b357-e59d6e981799 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:37.772125 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1a6f6e44-40e6-49f9-b357-e59d6e981799 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:37.921469 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-1a6f6e44-40e6-49f9-b357-e59d6e981799 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:37.929750 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-1a6f6e44-40e6-49f9-b357-e59d6e981799 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578" status: 200 len: 1365 microversion: 2.1 time: 0.173383 Aug 08 02:29:37.930532 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 275/549] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:37 2022] GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578 => generated 1365 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:38.265435 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-44e08b44-f241-44bc-8ebd-6cf9f9a719c5 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:38.266057 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-44e08b44-f241-44bc-8ebd-6cf9f9a719c5 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:38.272649 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-44e08b44-f241-44bc-8ebd-6cf9f9a719c5 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:38.314233 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-44e08b44-f241-44bc-8ebd-6cf9f9a719c5 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:38.314233 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-44e08b44-f241-44bc-8ebd-6cf9f9a719c5 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:38.314233 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-44e08b44-f241-44bc-8ebd-6cf9f9a719c5 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:38.314233 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-44e08b44-f241-44bc-8ebd-6cf9f9a719c5 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:38.314642 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-44e08b44-f241-44bc-8ebd-6cf9f9a719c5 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:38.404945 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-44e08b44-f241-44bc-8ebd-6cf9f9a719c5 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:38.412569 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-44e08b44-f241-44bc-8ebd-6cf9f9a719c5 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1259 microversion: 2.1 time: 0.151212 Aug 08 02:29:38.413163 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 275/550] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:38 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1259 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:38.952034 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-751a6064-407b-489b-8eb3-3e87ae17053c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:38.952642 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-751a6064-407b-489b-8eb3-3e87ae17053c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: aa20d25c-bd00-41e6-b350-309368839578] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:38.959758 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-751a6064-407b-489b-8eb3-3e87ae17053c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:38.960238 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-751a6064-407b-489b-8eb3-3e87ae17053c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:38.960687 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-751a6064-407b-489b-8eb3-3e87ae17053c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:38.961384 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-751a6064-407b-489b-8eb3-3e87ae17053c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:38.961839 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-751a6064-407b-489b-8eb3-3e87ae17053c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:38.962265 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-751a6064-407b-489b-8eb3-3e87ae17053c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:39.090891 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-751a6064-407b-489b-8eb3-3e87ae17053c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:39.098926 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-751a6064-407b-489b-8eb3-3e87ae17053c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578" status: 200 len: 1365 microversion: 2.1 time: 0.150683 Aug 08 02:29:39.102610 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 276/551] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:38 2022] GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578 => generated 1365 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:39.432428 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-6f0fcdb4-0098-4d7d-ad71-777c0edb2739 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:39.433211 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-6f0fcdb4-0098-4d7d-ad71-777c0edb2739 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:39.439782 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6f0fcdb4-0098-4d7d-ad71-777c0edb2739 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:39.444192 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6f0fcdb4-0098-4d7d-ad71-777c0edb2739 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:39.444192 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6f0fcdb4-0098-4d7d-ad71-777c0edb2739 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:39.444192 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6f0fcdb4-0098-4d7d-ad71-777c0edb2739 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:39.444192 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6f0fcdb4-0098-4d7d-ad71-777c0edb2739 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:39.444773 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6f0fcdb4-0098-4d7d-ad71-777c0edb2739 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:39.559030 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [req-8aac21b7-580d-486a-9f46-e1e2ac1310e8 req-c1817a95-5af7-446a-b0ca-4c6c40ec5218 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "aa20d25c-bd00-41e6-b350-309368839578", "name": "network-vif-plugged", "status": "completed", "tag": "72aebc3e-9c8c-4a19-8c44-a4ea8d03d79e"}]} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:29:39.567085 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-8aac21b7-580d-486a-9f46-e1e2ac1310e8 req-c1817a95-5af7-446a-b0ca-4c6c40ec5218 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:39.567849 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-8aac21b7-580d-486a-9f46-e1e2ac1310e8 req-c1817a95-5af7-446a-b0ca-4c6c40ec5218 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:39.568372 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-8aac21b7-580d-486a-9f46-e1e2ac1310e8 req-c1817a95-5af7-446a-b0ca-4c6c40ec5218 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:39.579699 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.compute.server_external_events [req-8aac21b7-580d-486a-9f46-e1e2ac1310e8 req-c1817a95-5af7-446a-b0ca-4c6c40ec5218 service nova] Creating event network-vif-plugged:72aebc3e-9c8c-4a19-8c44-a4ea8d03d79e for instance aa20d25c-bd00-41e6-b350-309368839578 on hv-cinder-74014 Aug 08 02:29:39.585643 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [req-8aac21b7-580d-486a-9f46-e1e2ac1310e8 req-c1817a95-5af7-446a-b0ca-4c6c40ec5218 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034426 Aug 08 02:29:39.591585 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-6f0fcdb4-0098-4d7d-ad71-777c0edb2739 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:39.592229 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 277/552] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:29:39 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:29:39.600810 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-6f0fcdb4-0098-4d7d-ad71-777c0edb2739 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1259 microversion: 2.1 time: 0.172052 Aug 08 02:29:39.601402 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 276/553] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:39 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1259 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:40.121494 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-e816dc9f-1cac-4719-b6bb-c03d8f489708 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:40.122184 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-e816dc9f-1cac-4719-b6bb-c03d8f489708 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: aa20d25c-bd00-41e6-b350-309368839578] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:40.128246 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e816dc9f-1cac-4719-b6bb-c03d8f489708 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:40.128752 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e816dc9f-1cac-4719-b6bb-c03d8f489708 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:40.129799 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e816dc9f-1cac-4719-b6bb-c03d8f489708 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:40.130486 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e816dc9f-1cac-4719-b6bb-c03d8f489708 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:40.130943 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e816dc9f-1cac-4719-b6bb-c03d8f489708 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:40.131368 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e816dc9f-1cac-4719-b6bb-c03d8f489708 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:40.272243 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-e816dc9f-1cac-4719-b6bb-c03d8f489708 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:40.279622 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-e816dc9f-1cac-4719-b6bb-c03d8f489708 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578" status: 200 len: 1365 microversion: 2.1 time: 0.162850 Aug 08 02:29:40.280185 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 278/554] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:40 2022] GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578 => generated 1365 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:40.621723 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-beef0c24-fdd5-4ead-bee6-f8f4bd1c1ae3 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:40.622345 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-beef0c24-fdd5-4ead-bee6-f8f4bd1c1ae3 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:40.628848 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-beef0c24-fdd5-4ead-bee6-f8f4bd1c1ae3 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:40.639077 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-beef0c24-fdd5-4ead-bee6-f8f4bd1c1ae3 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:40.639077 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-beef0c24-fdd5-4ead-bee6-f8f4bd1c1ae3 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:40.639077 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-beef0c24-fdd5-4ead-bee6-f8f4bd1c1ae3 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:40.639077 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-beef0c24-fdd5-4ead-bee6-f8f4bd1c1ae3 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:40.639435 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-beef0c24-fdd5-4ead-bee6-f8f4bd1c1ae3 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:40.763772 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-beef0c24-fdd5-4ead-bee6-f8f4bd1c1ae3 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:40.771973 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-beef0c24-fdd5-4ead-bee6-f8f4bd1c1ae3 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1259 microversion: 2.1 time: 0.154116 Aug 08 02:29:40.772538 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 277/555] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:40 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1259 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:41.301449 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-cfbcdf67-e930-46e9-8897-70b63e5322ab tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:41.302365 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-cfbcdf67-e930-46e9-8897-70b63e5322ab tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: aa20d25c-bd00-41e6-b350-309368839578] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:41.309049 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cfbcdf67-e930-46e9-8897-70b63e5322ab tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:41.309541 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cfbcdf67-e930-46e9-8897-70b63e5322ab tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:41.309980 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cfbcdf67-e930-46e9-8897-70b63e5322ab tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:41.310673 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cfbcdf67-e930-46e9-8897-70b63e5322ab tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:41.311132 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cfbcdf67-e930-46e9-8897-70b63e5322ab tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:41.311560 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cfbcdf67-e930-46e9-8897-70b63e5322ab tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:41.446076 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-cfbcdf67-e930-46e9-8897-70b63e5322ab tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:41.453501 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-cfbcdf67-e930-46e9-8897-70b63e5322ab tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578" status: 200 len: 1365 microversion: 2.1 time: 0.156176 Aug 08 02:29:41.454062 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 279/556] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:41 2022] GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578 => generated 1365 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:41.792515 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-6fab7210-cb47-49b7-9f06-da3723b0959f tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:41.793159 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-6fab7210-cb47-49b7-9f06-da3723b0959f tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:41.799683 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6fab7210-cb47-49b7-9f06-da3723b0959f tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:41.800161 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6fab7210-cb47-49b7-9f06-da3723b0959f tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:41.800610 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6fab7210-cb47-49b7-9f06-da3723b0959f tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:41.801743 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6fab7210-cb47-49b7-9f06-da3723b0959f tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:41.802196 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6fab7210-cb47-49b7-9f06-da3723b0959f tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:41.802635 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6fab7210-cb47-49b7-9f06-da3723b0959f tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:41.934497 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-6fab7210-cb47-49b7-9f06-da3723b0959f tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:41.942678 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-6fab7210-cb47-49b7-9f06-da3723b0959f tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1259 microversion: 2.1 time: 0.153134 Aug 08 02:29:41.944412 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 278/557] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:41 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1259 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:42.407012 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-b9ee73b5-0139-4e2e-8416-da565dffbe52 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:42.407577 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-b9ee73b5-0139-4e2e-8416-da565dffbe52 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: f2edabed-f896-4ecb-ab34-ef137c01d581] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:42.415277 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b9ee73b5-0139-4e2e-8416-da565dffbe52 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:42.415823 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b9ee73b5-0139-4e2e-8416-da565dffbe52 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:42.416292 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b9ee73b5-0139-4e2e-8416-da565dffbe52 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:42.417008 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b9ee73b5-0139-4e2e-8416-da565dffbe52 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:42.417461 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b9ee73b5-0139-4e2e-8416-da565dffbe52 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:42.417879 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b9ee73b5-0139-4e2e-8416-da565dffbe52 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:42.474713 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-2bc343bf-8e0d-4726-9488-c588b4102bf6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:42.475424 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-2bc343bf-8e0d-4726-9488-c588b4102bf6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: aa20d25c-bd00-41e6-b350-309368839578] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:42.481422 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2bc343bf-8e0d-4726-9488-c588b4102bf6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:42.481915 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2bc343bf-8e0d-4726-9488-c588b4102bf6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:42.482351 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2bc343bf-8e0d-4726-9488-c588b4102bf6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:42.483038 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2bc343bf-8e0d-4726-9488-c588b4102bf6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:42.483491 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2bc343bf-8e0d-4726-9488-c588b4102bf6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:42.483918 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2bc343bf-8e0d-4726-9488-c588b4102bf6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:42.503351 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.objects.instance [None req-b9ee73b5-0139-4e2e-8416-da565dffbe52 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lazy-loading 'services' on Instance uuid f2edabed-f896-4ecb-ab34-ef137c01d581 {{(pid=118352) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 08 02:29:42.616911 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-2bc343bf-8e0d-4726-9488-c588b4102bf6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:42.624604 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-2bc343bf-8e0d-4726-9488-c588b4102bf6 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578" status: 200 len: 1555 microversion: 2.1 time: 0.154185 Aug 08 02:29:42.627870 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 279/558] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:42 2022] GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578 => generated 1555 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:29:42.645417 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-0a3dcfdf-ac17-42a6-8536-cd6327be13df tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:42.645844 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-0a3dcfdf-ac17-42a6-8536-cd6327be13df tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: aa20d25c-bd00-41e6-b350-309368839578] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:42.651719 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0a3dcfdf-ac17-42a6-8536-cd6327be13df tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:42.658556 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0a3dcfdf-ac17-42a6-8536-cd6327be13df tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:42.658556 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0a3dcfdf-ac17-42a6-8536-cd6327be13df tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:42.658556 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0a3dcfdf-ac17-42a6-8536-cd6327be13df tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:42.658556 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0a3dcfdf-ac17-42a6-8536-cd6327be13df tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:42.658916 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0a3dcfdf-ac17-42a6-8536-cd6327be13df tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:42.710675 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-b9ee73b5-0139-4e2e-8416-da565dffbe52 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "DELETE /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581/os-volume_attachments/14f9aecb-ce30-4121-87dd-237586a72402" status: 202 len: 0 microversion: 2.1 time: 0.307528 Aug 08 02:29:42.711500 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 280/559] 10.222.0.27 () {58 vars in 1416 bytes} [Mon Aug 8 02:29:42 2022] DELETE /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581/os-volume_attachments/14f9aecb-ce30-4121-87dd-237586a72402 => generated 0 bytes in 309 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 08 02:29:42.734386 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-0a3dcfdf-ac17-42a6-8536-cd6327be13df tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.092910 Aug 08 02:29:42.734973 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 280/560] 10.222.0.27 () {58 vars in 1275 bytes} [Mon Aug 8 02:29:42 2022] GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578/os-interface => generated 284 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 08 02:29:42.964623 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-874b987d-195e-4174-947f-5fbd5e9f6e74 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:42.965263 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-874b987d-195e-4174-947f-5fbd5e9f6e74 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:42.971554 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-874b987d-195e-4174-947f-5fbd5e9f6e74 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:42.972112 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-874b987d-195e-4174-947f-5fbd5e9f6e74 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:42.972553 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-874b987d-195e-4174-947f-5fbd5e9f6e74 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:42.973945 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-874b987d-195e-4174-947f-5fbd5e9f6e74 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:42.974500 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-874b987d-195e-4174-947f-5fbd5e9f6e74 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:42.974923 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-874b987d-195e-4174-947f-5fbd5e9f6e74 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:43.226867 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-874b987d-195e-4174-947f-5fbd5e9f6e74 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:43.234636 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-874b987d-195e-4174-947f-5fbd5e9f6e74 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1259 microversion: 2.1 time: 0.274181 Aug 08 02:29:43.235209 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 281/561] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:42 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1259 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:43.810496 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [req-a8491600-c9ee-4d04-900b-de8d649b2520 req-e1483408-7d8d-4705-98c7-f609d8106607 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "aa20d25c-bd00-41e6-b350-309368839578", "tag": "72aebc3e-9c8c-4a19-8c44-a4ea8d03d79e"}]} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:29:43.820649 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-a8491600-c9ee-4d04-900b-de8d649b2520 req-e1483408-7d8d-4705-98c7-f609d8106607 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:43.821158 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-a8491600-c9ee-4d04-900b-de8d649b2520 req-e1483408-7d8d-4705-98c7-f609d8106607 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:43.821795 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-a8491600-c9ee-4d04-900b-de8d649b2520 req-e1483408-7d8d-4705-98c7-f609d8106607 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:43.834665 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.compute.server_external_events [req-a8491600-c9ee-4d04-900b-de8d649b2520 req-e1483408-7d8d-4705-98c7-f609d8106607 service nova] Creating event network-changed:72aebc3e-9c8c-4a19-8c44-a4ea8d03d79e for instance aa20d25c-bd00-41e6-b350-309368839578 on hv-cinder-74014 Aug 08 02:29:43.845794 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [req-a8491600-c9ee-4d04-900b-de8d649b2520 req-e1483408-7d8d-4705-98c7-f609d8106607 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034034 Aug 08 02:29:43.845794 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 281/562] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:29:43 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:29:44.258910 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-f196b288-84ec-4b3d-a9f4-57562139b124 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:44.259625 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-f196b288-84ec-4b3d-a9f4-57562139b124 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:44.265482 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f196b288-84ec-4b3d-a9f4-57562139b124 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:44.265963 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f196b288-84ec-4b3d-a9f4-57562139b124 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:44.266410 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f196b288-84ec-4b3d-a9f4-57562139b124 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:44.267080 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f196b288-84ec-4b3d-a9f4-57562139b124 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:44.267530 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f196b288-84ec-4b3d-a9f4-57562139b124 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:44.267950 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f196b288-84ec-4b3d-a9f4-57562139b124 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:44.405638 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-f196b288-84ec-4b3d-a9f4-57562139b124 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:44.414148 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-f196b288-84ec-4b3d-a9f4-57562139b124 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1259 microversion: 2.1 time: 0.159949 Aug 08 02:29:44.414728 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 282/563] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:44 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1259 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:44.923312 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-b91cf102-1a69-4c10-9bab-096af330b833 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:44.923986 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-b91cf102-1a69-4c10-9bab-096af330b833 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: f2edabed-f896-4ecb-ab34-ef137c01d581] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:44.930859 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b91cf102-1a69-4c10-9bab-096af330b833 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:44.931293 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b91cf102-1a69-4c10-9bab-096af330b833 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:44.931942 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b91cf102-1a69-4c10-9bab-096af330b833 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:44.932861 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b91cf102-1a69-4c10-9bab-096af330b833 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:44.933462 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b91cf102-1a69-4c10-9bab-096af330b833 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:44.937421 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b91cf102-1a69-4c10-9bab-096af330b833 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:44.960954 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-b91cf102-1a69-4c10-9bab-096af330b833 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: f2edabed-f896-4ecb-ab34-ef137c01d581] Going to try to terminate instance {{(pid=118351) delete /opt/stack/nova/nova/compute/api.py:2666}} Aug 08 02:29:45.288575 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-b91cf102-1a69-4c10-9bab-096af330b833 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "DELETE /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581" status: 204 len: 0 microversion: 2.1 time: 0.369696 Aug 08 02:29:45.289201 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 282/564] 10.222.0.27 () {58 vars in 1239 bytes} [Mon Aug 8 02:29:44 2022] DELETE /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 08 02:29:45.297395 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-1a33bb24-ffdb-455a-9bac-aa55ac45799a tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:45.298018 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-1a33bb24-ffdb-455a-9bac-aa55ac45799a tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: f2edabed-f896-4ecb-ab34-ef137c01d581] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:45.306377 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1a33bb24-ffdb-455a-9bac-aa55ac45799a tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:45.306867 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1a33bb24-ffdb-455a-9bac-aa55ac45799a tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:45.307313 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1a33bb24-ffdb-455a-9bac-aa55ac45799a tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:45.308008 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1a33bb24-ffdb-455a-9bac-aa55ac45799a tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:45.308473 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1a33bb24-ffdb-455a-9bac-aa55ac45799a tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:45.308961 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1a33bb24-ffdb-455a-9bac-aa55ac45799a tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:45.437044 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-c2f99144-40e9-4816-83c6-8cb43b5b45ba tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:45.437743 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-c2f99144-40e9-4816-83c6-8cb43b5b45ba tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:45.437866 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-1a33bb24-ffdb-455a-9bac-aa55ac45799a tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5e3c891f2f3148f4aaba41a14d507cdf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7203c52ae1f946bd9717a554316e4f4c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:45.444545 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c2f99144-40e9-4816-83c6-8cb43b5b45ba tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:45.445137 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c2f99144-40e9-4816-83c6-8cb43b5b45ba tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:45.445573 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c2f99144-40e9-4816-83c6-8cb43b5b45ba tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:45.446258 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c2f99144-40e9-4816-83c6-8cb43b5b45ba tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:45.446359 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-1a33bb24-ffdb-455a-9bac-aa55ac45799a tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581" status: 200 len: 1514 microversion: 2.1 time: 0.152743 Aug 08 02:29:45.446703 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c2f99144-40e9-4816-83c6-8cb43b5b45ba tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:45.447139 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 283/565] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:45 2022] GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581 => generated 1514 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:45.447261 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c2f99144-40e9-4816-83c6-8cb43b5b45ba tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:45.576616 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-c2f99144-40e9-4816-83c6-8cb43b5b45ba tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:45.584340 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-c2f99144-40e9-4816-83c6-8cb43b5b45ba tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1259 microversion: 2.1 time: 0.153030 Aug 08 02:29:45.584954 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 283/566] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:45 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1259 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:29:46.467166 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-9c652418-5b7c-48f9-8254-c815833d0d71 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:46.467781 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-9c652418-5b7c-48f9-8254-c815833d0d71 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: f2edabed-f896-4ecb-ab34-ef137c01d581] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:46.474731 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9c652418-5b7c-48f9-8254-c815833d0d71 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:46.475209 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9c652418-5b7c-48f9-8254-c815833d0d71 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:46.475650 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9c652418-5b7c-48f9-8254-c815833d0d71 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:46.476332 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9c652418-5b7c-48f9-8254-c815833d0d71 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:46.476822 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9c652418-5b7c-48f9-8254-c815833d0d71 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:46.477252 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9c652418-5b7c-48f9-8254-c815833d0d71 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:46.605178 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-47ebd8f3-d341-40fb-a519-ab5db6d11506 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:46.605873 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-47ebd8f3-d341-40fb-a519-ab5db6d11506 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:46.606812 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-9c652418-5b7c-48f9-8254-c815833d0d71 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5e3c891f2f3148f4aaba41a14d507cdf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7203c52ae1f946bd9717a554316e4f4c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:46.608599 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_api:SELECT=85,nova_api:DELETE=1,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:29:46.613477 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-47ebd8f3-d341-40fb-a519-ab5db6d11506 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:46.614497 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-47ebd8f3-d341-40fb-a519-ab5db6d11506 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:46.614935 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-47ebd8f3-d341-40fb-a519-ab5db6d11506 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:46.615427 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-9c652418-5b7c-48f9-8254-c815833d0d71 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581" status: 200 len: 1514 microversion: 2.1 time: 0.152095 Aug 08 02:29:46.615622 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-47ebd8f3-d341-40fb-a519-ab5db6d11506 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:46.616008 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 284/567] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:46 2022] GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581 => generated 1514 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:46.616134 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-47ebd8f3-d341-40fb-a519-ab5db6d11506 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:46.616483 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-47ebd8f3-d341-40fb-a519-ab5db6d11506 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:46.746207 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-47ebd8f3-d341-40fb-a519-ab5db6d11506 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:46.753829 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-47ebd8f3-d341-40fb-a519-ab5db6d11506 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1259 microversion: 2.1 time: 0.152691 Aug 08 02:29:46.755066 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 284/568] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:46 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1259 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:47.636574 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-1db165b2-7a71-4959-9b94-22a742eb2088 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:47.637238 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-1db165b2-7a71-4959-9b94-22a742eb2088 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: f2edabed-f896-4ecb-ab34-ef137c01d581] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:47.639651 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_api:SELECT=94,nova_api:INSERT=4 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:29:47.644281 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1db165b2-7a71-4959-9b94-22a742eb2088 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:47.644805 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1db165b2-7a71-4959-9b94-22a742eb2088 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:47.645883 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1db165b2-7a71-4959-9b94-22a742eb2088 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:47.646586 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1db165b2-7a71-4959-9b94-22a742eb2088 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:47.647041 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1db165b2-7a71-4959-9b94-22a742eb2088 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:47.647466 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1db165b2-7a71-4959-9b94-22a742eb2088 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:47.777635 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-d07b21df-4b95-441a-abf5-9ca7a97598e7 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:47.783774 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-d07b21df-4b95-441a-abf5-9ca7a97598e7 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:47.787126 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d07b21df-4b95-441a-abf5-9ca7a97598e7 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:47.787886 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d07b21df-4b95-441a-abf5-9ca7a97598e7 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:47.788755 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d07b21df-4b95-441a-abf5-9ca7a97598e7 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:47.789595 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d07b21df-4b95-441a-abf5-9ca7a97598e7 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:47.790272 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d07b21df-4b95-441a-abf5-9ca7a97598e7 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:47.790962 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d07b21df-4b95-441a-abf5-9ca7a97598e7 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:48.005390 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-1db165b2-7a71-4959-9b94-22a742eb2088 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5e3c891f2f3148f4aaba41a14d507cdf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7203c52ae1f946bd9717a554316e4f4c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:48.015511 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-1db165b2-7a71-4959-9b94-22a742eb2088 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581" status: 200 len: 1514 microversion: 2.1 time: 0.382922 Aug 08 02:29:48.018026 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 285/569] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:47 2022] GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581 => generated 1514 bytes in 385 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:48.023357 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [req-32f8e65f-0b3c-4a8a-82d4-d42ed6a77c5b req-cadf0d81-37d9-4ee7-a459-09b4d18faf93 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f2edabed-f896-4ecb-ab34-ef137c01d581", "name": "network-vif-deleted", "tag": "f553eec7-8945-4f14-8a23-9cc399094640"}]} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:29:48.033454 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-32f8e65f-0b3c-4a8a-82d4-d42ed6a77c5b req-cadf0d81-37d9-4ee7-a459-09b4d18faf93 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:48.034278 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-32f8e65f-0b3c-4a8a-82d4-d42ed6a77c5b req-cadf0d81-37d9-4ee7-a459-09b4d18faf93 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:48.034910 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-32f8e65f-0b3c-4a8a-82d4-d42ed6a77c5b req-cadf0d81-37d9-4ee7-a459-09b4d18faf93 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:48.051982 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.compute.server_external_events [req-32f8e65f-0b3c-4a8a-82d4-d42ed6a77c5b req-cadf0d81-37d9-4ee7-a459-09b4d18faf93 service nova] Creating event network-vif-deleted:f553eec7-8945-4f14-8a23-9cc399094640 for instance f2edabed-f896-4ecb-ab34-ef137c01d581 on hv-cinder-74014 Aug 08 02:29:48.057169 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [req-32f8e65f-0b3c-4a8a-82d4-d42ed6a77c5b req-cadf0d81-37d9-4ee7-a459-09b4d18faf93 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038478 Aug 08 02:29:48.057848 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 286/570] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:29:48 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:29:48.117435 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-d07b21df-4b95-441a-abf5-9ca7a97598e7 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:48.125048 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-d07b21df-4b95-441a-abf5-9ca7a97598e7 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1259 microversion: 2.1 time: 0.352582 Aug 08 02:29:48.125614 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 285/571] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:47 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1259 bytes in 354 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:49.038507 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-4792a31d-a15f-4d55-af66-4933f6978b28 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:49.039213 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-4792a31d-a15f-4d55-af66-4933f6978b28 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: f2edabed-f896-4ecb-ab34-ef137c01d581] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:49.045472 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4792a31d-a15f-4d55-af66-4933f6978b28 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:49.045955 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4792a31d-a15f-4d55-af66-4933f6978b28 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:49.046398 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4792a31d-a15f-4d55-af66-4933f6978b28 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:49.047089 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4792a31d-a15f-4d55-af66-4933f6978b28 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:49.047549 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4792a31d-a15f-4d55-af66-4933f6978b28 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:49.047976 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4792a31d-a15f-4d55-af66-4933f6978b28 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:49.062648 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.wsgi [None req-4792a31d-a15f-4d55-af66-4933f6978b28 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] HTTP exception thrown: Instance f2edabed-f896-4ecb-ab34-ef137c01d581 could not be found. Aug 08 02:29:49.064308 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-4792a31d-a15f-4d55-af66-4933f6978b28 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Request method failure captured: Aug 08 02:29:49.064308 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: request: GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581 HTTP/1.1 Aug 08 02:29:49.064308 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept: application/json Aug 08 02:29:49.064308 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept-Encoding: identity Aug 08 02:29:49.064308 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Connection: close Aug 08 02:29:49.064308 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Length: 0 Aug 08 02:29:49.064308 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Type: application/json Aug 08 02:29:49.064308 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Host: 10.222.0.27 Aug 08 02:29:49.064308 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Openstack-System-Scope: None Aug 08 02:29:49.064308 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: User-Agent: python-urllib3/1.26.11 Aug 08 02:29:49.064308 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Auth-Token: gAAAAABi8HTH8a0CBQ8lEnnKCKkC_gQfnY8t7Ey_gArOYoTtLvqT8lOtRp95WoRUY5Q1WdPf_EGbzi8cT3to_wING_-blz9aiXVpDzS907isS90uSr-YeThzVTF_ORbCFIluGwDnAYt15m-PmK6CNHU-RauOpu9WoCM938azTnTeZfmzXoi2yzo Aug 08 02:29:49.064308 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Id: None Aug 08 02:29:49.064308 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Name: None Aug 08 02:29:49.064308 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Identity-Status: Confirmed Aug 08 02:29:49.064308 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Is-Admin-Project: True Aug 08 02:29:49.064308 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Id: default Aug 08 02:29:49.064308 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Name: Default Aug 08 02:29:49.064308 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Id: 7203c52ae1f946bd9717a554316e4f4c Aug 08 02:29:49.064308 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Name: tempest-VolumesSnapshotTestJSON-71153930 Aug 08 02:29:49.064308 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Role: reader,member Aug 08 02:29:49.064308 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Roles: reader,member Aug 08 02:29:49.071360 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/7203c52ae1f946bd9717a554316e4f4c"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/7203c52ae1f946bd9717a554316e4f4c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_7203c52ae1f946bd9717a554316e4f4c", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/7203c52ae1f946bd9717a554316e4f4c"}]}] Aug 08 02:29:49.071360 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant: tempest-VolumesSnapshotTestJSON-71153930 Aug 08 02:29:49.071360 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Id: 7203c52ae1f946bd9717a554316e4f4c Aug 08 02:29:49.071360 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-71153930 Aug 08 02:29:49.071360 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User: tempest-VolumesSnapshotTestJSON-71153930-project Aug 08 02:29:49.071360 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Id: default Aug 08 02:29:49.071360 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Name: Default Aug 08 02:29:49.071360 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Id: 5e3c891f2f3148f4aaba41a14d507cdf Aug 08 02:29:49.071360 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Name: tempest-VolumesSnapshotTestJSON-71153930-project Aug 08 02:29:49.071360 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: method: > Aug 08 02:29:49.071360 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: exception: Instance f2edabed-f896-4ecb-ab34-ef137c01d581 could not be found. Aug 08 02:29:49.071360 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: action_args: {'id': 'f2edabed-f896-4ecb-ab34-ef137c01d581'} Aug 08 02:29:49.072277 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:29:49.072277 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:29:49.072277 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 08 02:29:49.072277 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 08 02:29:49.072277 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 08 02:29:49.072277 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 08 02:29:49.072277 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 08 02:29:49.072277 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 08 02:29:49.072277 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 08 02:29:49.072277 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 08 02:29:49.072277 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f2edabed-f896-4ecb-ab34-ef137c01d581 could not be found. Aug 08 02:29:49.072277 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:29:49.072277 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:29:49.072277 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:29:49.072277 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:29:49.072277 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:29:49.072277 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:29:49.072277 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:29:49.072277 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:29:49.072277 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:29:49.073734 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:29:49.073734 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 08 02:29:49.073734 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 08 02:29:49.073734 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 08 02:29:49.073734 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 08 02:29:49.073734 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 08 02:29:49.073734 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:29:49.073734 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f2edabed-f896-4ecb-ab34-ef137c01d581 could not be found. Aug 08 02:29:49.073734 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:29:49.073734 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:29:49.073734 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:29:49.073734 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:29:49.073734 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:29:49.073734 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:29:49.073734 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:29:49.073734 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:29:49.073734 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f2edabed-f896-4ecb-ab34-ef137c01d581 could not be found. Aug 08 02:29:49.073734 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:29:49.075111 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-4792a31d-a15f-4d55-af66-4933f6978b28 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Returning 404 to user: Instance f2edabed-f896-4ecb-ab34-ef137c01d581 could not be found. {{(pid=118352) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:29:49.075111 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-4792a31d-a15f-4d55-af66-4933f6978b28 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581" status: 404 len: 111 microversion: 2.1 time: 0.032811 Aug 08 02:29:49.075111 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 287/572] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:49 2022] GET /compute/v2.1/servers/f2edabed-f896-4ecb-ab34-ef137c01d581 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 08 02:29:49.146880 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-83998c9a-878a-4c5e-882f-90e42c6772cb tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:49.147648 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-83998c9a-878a-4c5e-882f-90e42c6772cb tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:49.154109 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-83998c9a-878a-4c5e-882f-90e42c6772cb tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:49.162177 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-83998c9a-878a-4c5e-882f-90e42c6772cb tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:49.162177 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-83998c9a-878a-4c5e-882f-90e42c6772cb tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:49.162177 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-83998c9a-878a-4c5e-882f-90e42c6772cb tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:49.162177 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-83998c9a-878a-4c5e-882f-90e42c6772cb tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:49.162617 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-83998c9a-878a-4c5e-882f-90e42c6772cb tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:49.174367 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:29:49.293084 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-83998c9a-878a-4c5e-882f-90e42c6772cb tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:49.300494 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-83998c9a-878a-4c5e-882f-90e42c6772cb tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1259 microversion: 2.1 time: 0.158082 Aug 08 02:29:49.305467 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 286/573] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:49 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1259 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:49.584128 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [req-51a60301-214c-4827-93fb-8dd3036f354b req-28ab53cd-6e90-4b36-a397-b307dc5aecb7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "077e7935-3361-4ff5-9367-ba162efac09c", "name": "network-vif-plugged", "status": "completed", "tag": "3051cabc-48a3-4055-ae74-0ae74aa73005"}]} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:29:49.594042 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-51a60301-214c-4827-93fb-8dd3036f354b req-28ab53cd-6e90-4b36-a397-b307dc5aecb7 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:49.594539 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-51a60301-214c-4827-93fb-8dd3036f354b req-28ab53cd-6e90-4b36-a397-b307dc5aecb7 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:49.595059 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-51a60301-214c-4827-93fb-8dd3036f354b req-28ab53cd-6e90-4b36-a397-b307dc5aecb7 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:49.608868 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.compute.server_external_events [req-51a60301-214c-4827-93fb-8dd3036f354b req-28ab53cd-6e90-4b36-a397-b307dc5aecb7 service nova] Creating event network-vif-plugged:3051cabc-48a3-4055-ae74-0ae74aa73005 for instance 077e7935-3361-4ff5-9367-ba162efac09c on hv-cinder-74014 Aug 08 02:29:49.617711 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [req-51a60301-214c-4827-93fb-8dd3036f354b req-28ab53cd-6e90-4b36-a397-b307dc5aecb7 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034044 Aug 08 02:29:49.617711 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 288/574] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:29:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:29:50.323456 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-8b25448f-eb3d-47e1-9ae9-4b8c01790c8d tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:50.324001 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-8b25448f-eb3d-47e1-9ae9-4b8c01790c8d tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:50.330638 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8b25448f-eb3d-47e1-9ae9-4b8c01790c8d tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:50.331112 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8b25448f-eb3d-47e1-9ae9-4b8c01790c8d tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:50.331540 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8b25448f-eb3d-47e1-9ae9-4b8c01790c8d tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:50.332221 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8b25448f-eb3d-47e1-9ae9-4b8c01790c8d tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:50.332689 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8b25448f-eb3d-47e1-9ae9-4b8c01790c8d tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:50.333322 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8b25448f-eb3d-47e1-9ae9-4b8c01790c8d tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:50.461373 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-8b25448f-eb3d-47e1-9ae9-4b8c01790c8d tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:50.469465 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-8b25448f-eb3d-47e1-9ae9-4b8c01790c8d tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1259 microversion: 2.1 time: 0.150240 Aug 08 02:29:50.470029 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 287/575] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:50 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1259 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:51.491520 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-3afbef8b-859c-49f4-8f8d-29f41e0c2df4 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:51.492211 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-3afbef8b-859c-49f4-8f8d-29f41e0c2df4 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:51.498387 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3afbef8b-859c-49f4-8f8d-29f41e0c2df4 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:51.498867 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3afbef8b-859c-49f4-8f8d-29f41e0c2df4 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:51.499299 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3afbef8b-859c-49f4-8f8d-29f41e0c2df4 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:51.499976 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3afbef8b-859c-49f4-8f8d-29f41e0c2df4 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:51.500424 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3afbef8b-859c-49f4-8f8d-29f41e0c2df4 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:51.500874 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3afbef8b-859c-49f4-8f8d-29f41e0c2df4 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:51.631876 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-3afbef8b-859c-49f4-8f8d-29f41e0c2df4 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:51.639306 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-3afbef8b-859c-49f4-8f8d-29f41e0c2df4 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1259 microversion: 2.1 time: 0.152620 Aug 08 02:29:51.639865 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 289/576] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:51 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1259 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:29:52.660153 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-61d7ccca-f772-489d-ad3c-bd3317ac681f tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:52.660819 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-61d7ccca-f772-489d-ad3c-bd3317ac681f tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:52.667811 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-61d7ccca-f772-489d-ad3c-bd3317ac681f tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:52.668288 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-61d7ccca-f772-489d-ad3c-bd3317ac681f tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:52.668750 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-61d7ccca-f772-489d-ad3c-bd3317ac681f tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:52.669425 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-61d7ccca-f772-489d-ad3c-bd3317ac681f tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:52.669871 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-61d7ccca-f772-489d-ad3c-bd3317ac681f tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:52.670287 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-61d7ccca-f772-489d-ad3c-bd3317ac681f tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:52.803523 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-61d7ccca-f772-489d-ad3c-bd3317ac681f tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:52.811220 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-61d7ccca-f772-489d-ad3c-bd3317ac681f tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1259 microversion: 2.1 time: 0.154948 Aug 08 02:29:52.811792 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 288/577] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:52 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1259 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:53.832313 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-591fcde7-e59e-4e1e-a0b5-b50468556e20 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:53.833000 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-591fcde7-e59e-4e1e-a0b5-b50468556e20 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:53.840192 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-591fcde7-e59e-4e1e-a0b5-b50468556e20 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:53.840614 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-591fcde7-e59e-4e1e-a0b5-b50468556e20 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:53.841086 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-591fcde7-e59e-4e1e-a0b5-b50468556e20 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:53.841780 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-591fcde7-e59e-4e1e-a0b5-b50468556e20 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:53.842236 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-591fcde7-e59e-4e1e-a0b5-b50468556e20 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:53.842670 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-591fcde7-e59e-4e1e-a0b5-b50468556e20 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:53.917597 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-81c97c5c-26b2-4e30-bbe4-0d111c1e41ba tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:53.918206 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-81c97c5c-26b2-4e30-bbe4-0d111c1e41ba tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:53.924787 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-81c97c5c-26b2-4e30-bbe4-0d111c1e41ba tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:53.925272 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-81c97c5c-26b2-4e30-bbe4-0d111c1e41ba tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:53.925703 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-81c97c5c-26b2-4e30-bbe4-0d111c1e41ba tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:53.926392 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-81c97c5c-26b2-4e30-bbe4-0d111c1e41ba tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:53.926844 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-81c97c5c-26b2-4e30-bbe4-0d111c1e41ba tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:53.927262 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-81c97c5c-26b2-4e30-bbe4-0d111c1e41ba tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:53.958602 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-81c97c5c-26b2-4e30-bbe4-0d111c1e41ba tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091/os-volume_attachments/5132860c-23b8-4d38-9a6c-3a0487900b41" status: 200 len: 194 microversion: 2.20 time: 0.045657 Aug 08 02:29:53.959201 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 289/578] 10.222.0.27 () {60 vars in 1454 bytes} [Mon Aug 8 02:29:53 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091/os-volume_attachments/5132860c-23b8-4d38-9a6c-3a0487900b41 => generated 194 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 08 02:29:53.979695 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-591fcde7-e59e-4e1e-a0b5-b50468556e20 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:53.990952 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-591fcde7-e59e-4e1e-a0b5-b50468556e20 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1259 microversion: 2.1 time: 0.162491 Aug 08 02:29:53.991520 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 290/579] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:53 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1259 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:54.031144 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-f5d43c8f-b676-48f1-bc6d-ecb09acf409f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:54.031643 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-f5d43c8f-b676-48f1-bc6d-ecb09acf409f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:54.037611 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f5d43c8f-b676-48f1-bc6d-ecb09acf409f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:54.038093 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f5d43c8f-b676-48f1-bc6d-ecb09acf409f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:54.039854 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f5d43c8f-b676-48f1-bc6d-ecb09acf409f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:54.040104 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f5d43c8f-b676-48f1-bc6d-ecb09acf409f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:54.040563 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f5d43c8f-b676-48f1-bc6d-ecb09acf409f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:54.041011 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f5d43c8f-b676-48f1-bc6d-ecb09acf409f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:54.120064 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.objects.instance [None req-f5d43c8f-b676-48f1-bc6d-ecb09acf409f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lazy-loading 'services' on Instance uuid f51a5900-5abc-4188-bbc4-33b0f0f7d091 {{(pid=118351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 08 02:29:54.431011 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-f5d43c8f-b676-48f1-bc6d-ecb09acf409f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "DELETE /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091/os-volume_attachments/5132860c-23b8-4d38-9a6c-3a0487900b41" status: 202 len: 0 microversion: 2.20 time: 0.403496 Aug 08 02:29:54.431535 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 290/580] 10.222.0.27 () {60 vars in 1457 bytes} [Mon Aug 8 02:29:54 2022] DELETE /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091/os-volume_attachments/5132860c-23b8-4d38-9a6c-3a0487900b41 => generated 0 bytes in 405 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Aug 08 02:29:54.440966 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-d141c022-9496-44a7-b7be-8f9b74b3937b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:54.443619 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-d141c022-9496-44a7-b7be-8f9b74b3937b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:54.451165 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d141c022-9496-44a7-b7be-8f9b74b3937b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:54.451664 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d141c022-9496-44a7-b7be-8f9b74b3937b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:54.452098 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d141c022-9496-44a7-b7be-8f9b74b3937b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:54.452832 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d141c022-9496-44a7-b7be-8f9b74b3937b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:54.453292 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d141c022-9496-44a7-b7be-8f9b74b3937b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:54.453727 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d141c022-9496-44a7-b7be-8f9b74b3937b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:54.483069 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-d141c022-9496-44a7-b7be-8f9b74b3937b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046946 Aug 08 02:29:54.483657 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 291/581] 10.222.0.27 () {60 vars in 1343 bytes} [Mon Aug 8 02:29:54 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 08 02:29:55.012348 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-e4149e96-32bc-426f-9b84-5894c4029a2c tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:55.013081 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-e4149e96-32bc-426f-9b84-5894c4029a2c tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:55.020296 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e4149e96-32bc-426f-9b84-5894c4029a2c tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:55.020943 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e4149e96-32bc-426f-9b84-5894c4029a2c tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:55.021795 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e4149e96-32bc-426f-9b84-5894c4029a2c tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:55.022593 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e4149e96-32bc-426f-9b84-5894c4029a2c tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:55.023409 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e4149e96-32bc-426f-9b84-5894c4029a2c tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:55.023601 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e4149e96-32bc-426f-9b84-5894c4029a2c tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:55.170155 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-e4149e96-32bc-426f-9b84-5894c4029a2c tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:55.177721 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-e4149e96-32bc-426f-9b84-5894c4029a2c tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1442 microversion: 2.1 time: 0.170091 Aug 08 02:29:55.178288 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 291/582] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:55 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1442 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:55.198794 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-ab6ddcf0-0f82-4552-b850-333e08239427 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:55.199466 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-ab6ddcf0-0f82-4552-b850-333e08239427 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:55.205972 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ab6ddcf0-0f82-4552-b850-333e08239427 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:55.206458 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ab6ddcf0-0f82-4552-b850-333e08239427 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:55.206896 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ab6ddcf0-0f82-4552-b850-333e08239427 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:55.207584 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ab6ddcf0-0f82-4552-b850-333e08239427 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:55.208042 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ab6ddcf0-0f82-4552-b850-333e08239427 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:55.208470 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ab6ddcf0-0f82-4552-b850-333e08239427 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:55.348981 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-ab6ddcf0-0f82-4552-b850-333e08239427 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:55.357805 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-ab6ddcf0-0f82-4552-b850-333e08239427 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1442 microversion: 2.1 time: 0.162898 Aug 08 02:29:55.358372 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 292/583] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:29:55 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1442 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:55.495746 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-dc46036e-aef0-4e5d-a7eb-61c9dff2cf9d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:55.498186 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-dc46036e-aef0-4e5d-a7eb-61c9dff2cf9d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:55.504402 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-dc46036e-aef0-4e5d-a7eb-61c9dff2cf9d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:55.504842 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-dc46036e-aef0-4e5d-a7eb-61c9dff2cf9d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:55.505454 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-dc46036e-aef0-4e5d-a7eb-61c9dff2cf9d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:55.506137 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-dc46036e-aef0-4e5d-a7eb-61c9dff2cf9d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:55.506583 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-dc46036e-aef0-4e5d-a7eb-61c9dff2cf9d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:55.506999 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-dc46036e-aef0-4e5d-a7eb-61c9dff2cf9d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:55.543905 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-dc46036e-aef0-4e5d-a7eb-61c9dff2cf9d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052253 Aug 08 02:29:55.548387 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 292/584] 10.222.0.27 () {60 vars in 1343 bytes} [Mon Aug 8 02:29:55 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 08 02:29:56.556200 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-7dc5b43a-aaa6-41ff-98ee-214c268a12d5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:56.558378 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-7dc5b43a-aaa6-41ff-98ee-214c268a12d5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:56.564917 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7dc5b43a-aaa6-41ff-98ee-214c268a12d5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:56.565405 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7dc5b43a-aaa6-41ff-98ee-214c268a12d5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:56.565845 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7dc5b43a-aaa6-41ff-98ee-214c268a12d5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:56.566544 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7dc5b43a-aaa6-41ff-98ee-214c268a12d5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:56.566998 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7dc5b43a-aaa6-41ff-98ee-214c268a12d5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:56.567422 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7dc5b43a-aaa6-41ff-98ee-214c268a12d5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:56.571752 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:29:56.603101 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-7dc5b43a-aaa6-41ff-98ee-214c268a12d5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.051513 Aug 08 02:29:56.603667 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 293/585] 10.222.0.27 () {60 vars in 1343 bytes} [Mon Aug 8 02:29:56 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091/os-volume_attachments => generated 25 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:29:56.669607 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-586bead6-24a6-42ed-bed6-b9d9f29d8dc7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:56.670171 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-586bead6-24a6-42ed-bed6-b9d9f29d8dc7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:56.676712 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-586bead6-24a6-42ed-bed6-b9d9f29d8dc7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:56.677203 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-586bead6-24a6-42ed-bed6-b9d9f29d8dc7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:56.677642 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-586bead6-24a6-42ed-bed6-b9d9f29d8dc7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:56.678352 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-586bead6-24a6-42ed-bed6-b9d9f29d8dc7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:56.678788 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-586bead6-24a6-42ed-bed6-b9d9f29d8dc7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:56.679212 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-586bead6-24a6-42ed-bed6-b9d9f29d8dc7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:56.705532 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-586bead6-24a6-42ed-bed6-b9d9f29d8dc7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Going to try to terminate instance {{(pid=118351) delete /opt/stack/nova/nova/compute/api.py:2666}} Aug 08 02:29:57.117524 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-586bead6-24a6-42ed-bed6-b9d9f29d8dc7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "DELETE /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 204 len: 0 microversion: 2.20 time: 0.450879 Aug 08 02:29:57.118260 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 293/586] 10.222.0.27 () {60 vars in 1280 bytes} [Mon Aug 8 02:29:56 2022] DELETE /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Aug 08 02:29:57.126728 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-afa0aa6a-511e-4898-b43a-63df9645cbf1 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:57.127343 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-afa0aa6a-511e-4898-b43a-63df9645cbf1 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:57.134896 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-afa0aa6a-511e-4898-b43a-63df9645cbf1 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:57.135388 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-afa0aa6a-511e-4898-b43a-63df9645cbf1 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:57.145895 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-afa0aa6a-511e-4898-b43a-63df9645cbf1 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:57.145895 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-afa0aa6a-511e-4898-b43a-63df9645cbf1 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:57.145895 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-afa0aa6a-511e-4898-b43a-63df9645cbf1 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:57.145895 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-afa0aa6a-511e-4898-b43a-63df9645cbf1 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:57.327410 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-afa0aa6a-511e-4898-b43a-63df9645cbf1 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:57.335376 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-afa0aa6a-511e-4898-b43a-63df9645cbf1 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:57.337673 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-afa0aa6a-511e-4898-b43a-63df9645cbf1 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:57.338821 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-afa0aa6a-511e-4898-b43a-63df9645cbf1 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1725 microversion: 2.20 time: 0.216143 Aug 08 02:29:57.339569 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 294/587] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:29:57 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1725 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:29:57.595143 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [req-048f7ba5-048a-4cbd-b1a4-89695b1348a5 req-fb369c39-5fc2-4bbb-8e21-b1ce9982f8e1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "077e7935-3361-4ff5-9367-ba162efac09c", "tag": "3051cabc-48a3-4055-ae74-0ae74aa73005"}]} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:29:57.606414 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-048f7ba5-048a-4cbd-b1a4-89695b1348a5 req-fb369c39-5fc2-4bbb-8e21-b1ce9982f8e1 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:57.606985 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-048f7ba5-048a-4cbd-b1a4-89695b1348a5 req-fb369c39-5fc2-4bbb-8e21-b1ce9982f8e1 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:57.607482 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-048f7ba5-048a-4cbd-b1a4-89695b1348a5 req-fb369c39-5fc2-4bbb-8e21-b1ce9982f8e1 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:57.621211 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.compute.server_external_events [req-048f7ba5-048a-4cbd-b1a4-89695b1348a5 req-fb369c39-5fc2-4bbb-8e21-b1ce9982f8e1 service nova] Creating event network-changed:3051cabc-48a3-4055-ae74-0ae74aa73005 for instance 077e7935-3361-4ff5-9367-ba162efac09c on hv-cinder-74014 Aug 08 02:29:57.626220 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [req-048f7ba5-048a-4cbd-b1a4-89695b1348a5 req-fb369c39-5fc2-4bbb-8e21-b1ce9982f8e1 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035442 Aug 08 02:29:57.626924 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 294/588] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:29:57 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:29:58.363141 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-e875bd97-f05e-4fc2-86be-a8254cbb471c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:58.363752 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-e875bd97-f05e-4fc2-86be-a8254cbb471c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:58.370832 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e875bd97-f05e-4fc2-86be-a8254cbb471c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:58.371312 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e875bd97-f05e-4fc2-86be-a8254cbb471c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:58.371746 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e875bd97-f05e-4fc2-86be-a8254cbb471c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:58.372426 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e875bd97-f05e-4fc2-86be-a8254cbb471c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:58.372895 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e875bd97-f05e-4fc2-86be-a8254cbb471c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:58.373319 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e875bd97-f05e-4fc2-86be-a8254cbb471c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:58.506641 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-e875bd97-f05e-4fc2-86be-a8254cbb471c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:58.515342 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-e875bd97-f05e-4fc2-86be-a8254cbb471c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:58.516860 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-e875bd97-f05e-4fc2-86be-a8254cbb471c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:58.518002 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-e875bd97-f05e-4fc2-86be-a8254cbb471c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1725 microversion: 2.20 time: 0.159055 Aug 08 02:29:58.518556 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 295/589] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:29:58 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1725 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:29:59.542197 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-a82a812d-9f41-444a-81ee-8f5390fb5e60 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:29:59.542893 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-a82a812d-9f41-444a-81ee-8f5390fb5e60 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:29:59.549045 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a82a812d-9f41-444a-81ee-8f5390fb5e60 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:59.549522 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a82a812d-9f41-444a-81ee-8f5390fb5e60 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:59.549954 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a82a812d-9f41-444a-81ee-8f5390fb5e60 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:59.550633 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a82a812d-9f41-444a-81ee-8f5390fb5e60 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:29:59.551081 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a82a812d-9f41-444a-81ee-8f5390fb5e60 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:29:59.551499 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a82a812d-9f41-444a-81ee-8f5390fb5e60 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:29:59.685904 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-a82a812d-9f41-444a-81ee-8f5390fb5e60 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:59.694326 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-a82a812d-9f41-444a-81ee-8f5390fb5e60 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:59.695884 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-a82a812d-9f41-444a-81ee-8f5390fb5e60 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:29:59.697866 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-a82a812d-9f41-444a-81ee-8f5390fb5e60 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1725 microversion: 2.20 time: 0.161127 Aug 08 02:29:59.698422 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 295/590] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:29:59 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1725 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 08 02:29:59.772631 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-266f5d28-93a5-4fd1-b298-49f07c1d5b9d tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-2078053169", "imageRef": "30833f47-5987-41ca-b5ee-ad5ecafbb265", "flavorRef": "452", "networks": [{"uuid": "80d55d70-3b18-4f03-aa9c-500250bb4b23"}]}} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:29:59.828424 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.network.neutron [None req-266f5d28-93a5-4fd1-b298-49f07c1d5b9d tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] validate_networks() for [('80d55d70-3b18-4f03-aa9c-500250bb4b23', None, None, None, None, None)] {{(pid=118352) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 08 02:30:00.172903 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-266f5d28-93a5-4fd1-b298-49f07c1d5b9d tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Flavor limits 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 08 02:30:00.177554 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-266f5d28-93a5-4fd1-b298-49f07c1d5b9d tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Image limits 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 08 02:30:00.177554 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-266f5d28-93a5-4fd1-b298-49f07c1d5b9d tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Flavor pref 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 08 02:30:00.177554 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-266f5d28-93a5-4fd1-b298-49f07c1d5b9d tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Image pref 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 08 02:30:00.177554 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-266f5d28-93a5-4fd1-b298-49f07c1d5b9d tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 08 02:30:00.184161 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-266f5d28-93a5-4fd1-b298-49f07c1d5b9d tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:00.184642 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-266f5d28-93a5-4fd1-b298-49f07c1d5b9d tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:00.185107 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-266f5d28-93a5-4fd1-b298-49f07c1d5b9d tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:00.185881 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-266f5d28-93a5-4fd1-b298-49f07c1d5b9d tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:00.186290 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-266f5d28-93a5-4fd1-b298-49f07c1d5b9d tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:00.186717 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-266f5d28-93a5-4fd1-b298-49f07c1d5b9d tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:00.204241 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.quota [None req-266f5d28-93a5-4fd1-b298-49f07c1d5b9d tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Getting quotas for project 7203c52ae1f946bd9717a554316e4f4c. Resources: {'ram', 'cores', 'instances'} {{(pid=118352) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 08 02:30:00.208347 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.quota [None req-266f5d28-93a5-4fd1-b298-49f07c1d5b9d tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Getting quotas for user 5e3c891f2f3148f4aaba41a14d507cdf and project 7203c52ae1f946bd9717a554316e4f4c. Resources: {'ram', 'cores', 'instances'} {{(pid=118352) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 08 02:30:00.219679 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-266f5d28-93a5-4fd1-b298-49f07c1d5b9d tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Going to run 1 instances... {{(pid=118352) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Aug 08 02:30:00.221984 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-266f5d28-93a5-4fd1-b298-49f07c1d5b9d tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: bd0f42d4-7cae-4736-8c36-fe5fa144908e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='30833f47-5987-41ca-b5ee-ad5ecafbb265',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118352) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Aug 08 02:30:00.340548 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-266f5d28-93a5-4fd1-b298-49f07c1d5b9d tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.571722 Aug 08 02:30:00.341227 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 296/591] 10.222.0.27 () {60 vars in 1147 bytes} [Mon Aug 8 02:29:59 2022] POST /compute/v2.1/servers => generated 380 bytes in 573 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 08 02:30:00.351852 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-9b8d677b-1a8e-400f-917f-6f48793b5b2d tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:00.353022 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-9b8d677b-1a8e-400f-917f-6f48793b5b2d tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: bd0f42d4-7cae-4736-8c36-fe5fa144908e] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:00.405698 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-9b8d677b-1a8e-400f-917f-6f48793b5b2d tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5e3c891f2f3148f4aaba41a14d507cdf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7203c52ae1f946bd9717a554316e4f4c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:00.412509 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-9b8d677b-1a8e-400f-917f-6f48793b5b2d tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "GET /compute/v2.1/servers/bd0f42d4-7cae-4736-8c36-fe5fa144908e" status: 200 len: 1219 microversion: 2.1 time: 0.064458 Aug 08 02:30:00.413116 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 296/592] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:00 2022] GET /compute/v2.1/servers/bd0f42d4-7cae-4736-8c36-fe5fa144908e => generated 1219 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:00.720946 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-e26c944f-638f-4e3c-94d3-3a9ea0c1ecc5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:00.721655 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-e26c944f-638f-4e3c-94d3-3a9ea0c1ecc5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:00.729240 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e26c944f-638f-4e3c-94d3-3a9ea0c1ecc5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:00.729818 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e26c944f-638f-4e3c-94d3-3a9ea0c1ecc5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:00.730336 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e26c944f-638f-4e3c-94d3-3a9ea0c1ecc5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:00.731110 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e26c944f-638f-4e3c-94d3-3a9ea0c1ecc5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:00.731740 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e26c944f-638f-4e3c-94d3-3a9ea0c1ecc5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:00.732345 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e26c944f-638f-4e3c-94d3-3a9ea0c1ecc5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:00.872250 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-e26c944f-638f-4e3c-94d3-3a9ea0c1ecc5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:00.880706 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-e26c944f-638f-4e3c-94d3-3a9ea0c1ecc5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:00.883522 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-e26c944f-638f-4e3c-94d3-3a9ea0c1ecc5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:00.883522 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-e26c944f-638f-4e3c-94d3-3a9ea0c1ecc5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1725 microversion: 2.20 time: 0.166457 Aug 08 02:30:00.884120 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 297/593] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:30:00 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1725 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:30:01.435508 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-4ee1768a-625d-489f-bbdd-b88d68d8c8e2 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:01.436135 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-4ee1768a-625d-489f-bbdd-b88d68d8c8e2 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: bd0f42d4-7cae-4736-8c36-fe5fa144908e] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:01.443685 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4ee1768a-625d-489f-bbdd-b88d68d8c8e2 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:01.444181 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4ee1768a-625d-489f-bbdd-b88d68d8c8e2 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:01.444619 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4ee1768a-625d-489f-bbdd-b88d68d8c8e2 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:01.445359 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4ee1768a-625d-489f-bbdd-b88d68d8c8e2 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:01.446617 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4ee1768a-625d-489f-bbdd-b88d68d8c8e2 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:01.447269 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4ee1768a-625d-489f-bbdd-b88d68d8c8e2 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:01.511276 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-4ee1768a-625d-489f-bbdd-b88d68d8c8e2 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5e3c891f2f3148f4aaba41a14d507cdf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7203c52ae1f946bd9717a554316e4f4c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:01.519150 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-4ee1768a-625d-489f-bbdd-b88d68d8c8e2 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "GET /compute/v2.1/servers/bd0f42d4-7cae-4736-8c36-fe5fa144908e" status: 200 len: 1223 microversion: 2.1 time: 0.088542 Aug 08 02:30:01.519884 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 297/594] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:01 2022] GET /compute/v2.1/servers/bd0f42d4-7cae-4736-8c36-fe5fa144908e => generated 1223 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:01.907641 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-40b39513-32be-4c00-9e16-4489dfd7f680 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:01.908394 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-40b39513-32be-4c00-9e16-4489dfd7f680 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:01.919096 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-40b39513-32be-4c00-9e16-4489dfd7f680 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:01.919096 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-40b39513-32be-4c00-9e16-4489dfd7f680 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:01.919096 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-40b39513-32be-4c00-9e16-4489dfd7f680 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:01.919096 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-40b39513-32be-4c00-9e16-4489dfd7f680 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:01.919541 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-40b39513-32be-4c00-9e16-4489dfd7f680 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:01.920034 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-40b39513-32be-4c00-9e16-4489dfd7f680 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:01.929342 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [req-24237a84-a030-45e1-84c8-4b1851c48998 req-1c31a8ef-85c1-4e9d-b776-eebdc28a574d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f51a5900-5abc-4188-bbc4-33b0f0f7d091", "name": "network-vif-deleted", "tag": "8d2a6aac-ff75-417c-b0b9-cbf799f88d22"}]} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:30:01.941339 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-24237a84-a030-45e1-84c8-4b1851c48998 req-1c31a8ef-85c1-4e9d-b776-eebdc28a574d service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:01.942101 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-24237a84-a030-45e1-84c8-4b1851c48998 req-1c31a8ef-85c1-4e9d-b776-eebdc28a574d service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:01.942691 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-24237a84-a030-45e1-84c8-4b1851c48998 req-1c31a8ef-85c1-4e9d-b776-eebdc28a574d service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:01.960166 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.compute.server_external_events [req-24237a84-a030-45e1-84c8-4b1851c48998 req-1c31a8ef-85c1-4e9d-b776-eebdc28a574d service nova] Creating event network-vif-deleted:8d2a6aac-ff75-417c-b0b9-cbf799f88d22 for instance f51a5900-5abc-4188-bbc4-33b0f0f7d091 on hv-cinder-74014 Aug 08 02:30:01.965309 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [req-24237a84-a030-45e1-84c8-4b1851c48998 req-1c31a8ef-85c1-4e9d-b776-eebdc28a574d service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039835 Aug 08 02:30:01.965917 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 298/595] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:30:01 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 08 02:30:01.992985 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-40b39513-32be-4c00-9e16-4489dfd7f680 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:02.010842 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-40b39513-32be-4c00-9e16-4489dfd7f680 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:02.010842 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-40b39513-32be-4c00-9e16-4489dfd7f680 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:02.010842 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-40b39513-32be-4c00-9e16-4489dfd7f680 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1657 microversion: 2.20 time: 0.106413 Aug 08 02:30:02.011752 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 298/596] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:30:01 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1657 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:30:02.540814 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-2caf965c-b9b7-40cf-9260-4eac5da5d132 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:02.541563 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-2caf965c-b9b7-40cf-9260-4eac5da5d132 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: bd0f42d4-7cae-4736-8c36-fe5fa144908e] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:02.548481 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2caf965c-b9b7-40cf-9260-4eac5da5d132 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:02.549312 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2caf965c-b9b7-40cf-9260-4eac5da5d132 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:02.549839 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2caf965c-b9b7-40cf-9260-4eac5da5d132 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:02.550608 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2caf965c-b9b7-40cf-9260-4eac5da5d132 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:02.551136 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2caf965c-b9b7-40cf-9260-4eac5da5d132 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:02.551770 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2caf965c-b9b7-40cf-9260-4eac5da5d132 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:02.625311 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-2caf965c-b9b7-40cf-9260-4eac5da5d132 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5e3c891f2f3148f4aaba41a14d507cdf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7203c52ae1f946bd9717a554316e4f4c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:02.633051 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-2caf965c-b9b7-40cf-9260-4eac5da5d132 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "GET /compute/v2.1/servers/bd0f42d4-7cae-4736-8c36-fe5fa144908e" status: 200 len: 1279 microversion: 2.1 time: 0.097241 Aug 08 02:30:02.635016 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 299/597] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:02 2022] GET /compute/v2.1/servers/bd0f42d4-7cae-4736-8c36-fe5fa144908e => generated 1279 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:03.035253 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-4afde0d5-9003-4f91-8936-a8d624748a35 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:03.035849 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-4afde0d5-9003-4f91-8936-a8d624748a35 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:03.043148 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4afde0d5-9003-4f91-8936-a8d624748a35 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:03.043637 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4afde0d5-9003-4f91-8936-a8d624748a35 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:03.047828 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4afde0d5-9003-4f91-8936-a8d624748a35 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:03.047828 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4afde0d5-9003-4f91-8936-a8d624748a35 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:03.047828 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4afde0d5-9003-4f91-8936-a8d624748a35 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:03.047828 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4afde0d5-9003-4f91-8936-a8d624748a35 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:03.082999 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4afde0d5-9003-4f91-8936-a8d624748a35 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:03.082999 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4afde0d5-9003-4f91-8936-a8d624748a35 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:03.082999 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4afde0d5-9003-4f91-8936-a8d624748a35 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:03.082999 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.objects.instance [None req-4afde0d5-9003-4f91-8936-a8d624748a35 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lazy-loading 'fault' on Instance uuid f51a5900-5abc-4188-bbc4-33b0f0f7d091 {{(pid=118352) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 08 02:30:03.133039 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-4afde0d5-9003-4f91-8936-a8d624748a35 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:03.145861 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-4afde0d5-9003-4f91-8936-a8d624748a35 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:03.146746 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-4afde0d5-9003-4f91-8936-a8d624748a35 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:03.147941 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-4afde0d5-9003-4f91-8936-a8d624748a35 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 200 len: 1372 microversion: 2.20 time: 0.116815 Aug 08 02:30:03.148503 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 299/598] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:30:03 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 1372 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:30:03.656278 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-c17ac887-16a0-4fa4-8027-b314fba2005e tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:03.656994 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-c17ac887-16a0-4fa4-8027-b314fba2005e tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: bd0f42d4-7cae-4736-8c36-fe5fa144908e] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:03.663636 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c17ac887-16a0-4fa4-8027-b314fba2005e tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:03.664115 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c17ac887-16a0-4fa4-8027-b314fba2005e tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:03.664554 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c17ac887-16a0-4fa4-8027-b314fba2005e tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:03.665833 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c17ac887-16a0-4fa4-8027-b314fba2005e tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:03.666285 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c17ac887-16a0-4fa4-8027-b314fba2005e tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:03.666708 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c17ac887-16a0-4fa4-8027-b314fba2005e tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:03.795264 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-c17ac887-16a0-4fa4-8027-b314fba2005e tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5e3c891f2f3148f4aaba41a14d507cdf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7203c52ae1f946bd9717a554316e4f4c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:03.804519 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-c17ac887-16a0-4fa4-8027-b314fba2005e tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "GET /compute/v2.1/servers/bd0f42d4-7cae-4736-8c36-fe5fa144908e" status: 200 len: 1319 microversion: 2.1 time: 0.151978 Aug 08 02:30:03.805199 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 300/599] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:03 2022] GET /compute/v2.1/servers/bd0f42d4-7cae-4736-8c36-fe5fa144908e => generated 1319 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:04.173108 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-b9759340-b614-49fa-9dd2-decbeafb1187 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:04.173729 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-b9759340-b614-49fa-9dd2-decbeafb1187 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:04.181056 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b9759340-b614-49fa-9dd2-decbeafb1187 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:04.181543 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b9759340-b614-49fa-9dd2-decbeafb1187 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:04.182431 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b9759340-b614-49fa-9dd2-decbeafb1187 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:04.183313 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b9759340-b614-49fa-9dd2-decbeafb1187 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:04.183939 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b9759340-b614-49fa-9dd2-decbeafb1187 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:04.184517 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b9759340-b614-49fa-9dd2-decbeafb1187 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:04.186462 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [req-2a28f5da-4fc8-40fe-a716-90aaab7b688c req-a57ac382-edc7-4ee0-a79c-b726e1e2910f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "bd0f42d4-7cae-4736-8c36-fe5fa144908e", "tag": "45cf1df5-f1d8-432f-8520-f0edf1de75df"}]} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:30:04.197754 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-2a28f5da-4fc8-40fe-a716-90aaab7b688c req-a57ac382-edc7-4ee0-a79c-b726e1e2910f service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:04.198238 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-2a28f5da-4fc8-40fe-a716-90aaab7b688c req-a57ac382-edc7-4ee0-a79c-b726e1e2910f service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:04.198831 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-2a28f5da-4fc8-40fe-a716-90aaab7b688c req-a57ac382-edc7-4ee0-a79c-b726e1e2910f service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:04.200148 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.wsgi [None req-b9759340-b614-49fa-9dd2-decbeafb1187 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] HTTP exception thrown: Instance f51a5900-5abc-4188-bbc4-33b0f0f7d091 could not be found. Aug 08 02:30:04.201818 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-b9759340-b614-49fa-9dd2-decbeafb1187 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Request method failure captured: Aug 08 02:30:04.201818 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: request: GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 HTTP/1.1 Aug 08 02:30:04.201818 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept: application/json Aug 08 02:30:04.201818 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept-Encoding: identity Aug 08 02:30:04.201818 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Connection: close Aug 08 02:30:04.201818 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Length: 0 Aug 08 02:30:04.201818 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Type: application/json Aug 08 02:30:04.201818 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Host: 10.222.0.27 Aug 08 02:30:04.201818 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Openstack-System-Scope: None Aug 08 02:30:04.201818 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: User-Agent: python-urllib3/1.26.11 Aug 08 02:30:04.201818 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Auth-Token: gAAAAABi8HQZM1u28QkO94W5_BI9_CK6trquSemX1B1L_Z-hB5Xr_ijBdmU9-G9fQkpWPZoqRyWzMs8DIWVoaDURV27YVJYuW-ZoqYszcoIWuLgl1q8SendZ4PZ3TqxVgh6HOFphSeT7fOMCYwtBnDGaMK-ygmFXZVrpyqVNef3pWtqKHJNgdaU Aug 08 02:30:04.201818 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Id: None Aug 08 02:30:04.201818 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Name: None Aug 08 02:30:04.201818 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Identity-Status: Confirmed Aug 08 02:30:04.201818 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Is-Admin-Project: True Aug 08 02:30:04.201818 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Openstack-Nova-Api-Version: 2.20 Aug 08 02:30:04.201818 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Id: default Aug 08 02:30:04.201818 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Name: Default Aug 08 02:30:04.201818 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Id: 7caa43bf43ad4e84b4b648e9ab33bd7f Aug 08 02:30:04.201818 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1887360550 Aug 08 02:30:04.201818 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Role: reader,member Aug 08 02:30:04.201818 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Roles: reader,member Aug 08 02:30:04.203293 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/7caa43bf43ad4e84b4b648e9ab33bd7f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/7caa43bf43ad4e84b4b648e9ab33bd7f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_7caa43bf43ad4e84b4b648e9ab33bd7f", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/7caa43bf43ad4e84b4b648e9ab33bd7f"}]}] Aug 08 02:30:04.203293 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1887360550 Aug 08 02:30:04.203293 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Id: 7caa43bf43ad4e84b4b648e9ab33bd7f Aug 08 02:30:04.203293 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1887360550 Aug 08 02:30:04.203293 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User: tempest-AttachVolumeShelveTestJSON-1887360550-project Aug 08 02:30:04.203293 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Id: default Aug 08 02:30:04.203293 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Name: Default Aug 08 02:30:04.203293 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Id: d451fb2f26ce4f50b010fb2a387c4bf3 Aug 08 02:30:04.203293 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1887360550-project Aug 08 02:30:04.203293 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: method: > Aug 08 02:30:04.203293 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: exception: Instance f51a5900-5abc-4188-bbc4-33b0f0f7d091 could not be found. Aug 08 02:30:04.203293 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: action_args: {'id': 'f51a5900-5abc-4188-bbc4-33b0f0f7d091'} Aug 08 02:30:04.204074 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:30:04.204074 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:30:04.204074 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 08 02:30:04.204074 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 08 02:30:04.204074 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 08 02:30:04.204074 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 08 02:30:04.204074 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 08 02:30:04.204074 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 08 02:30:04.204074 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 08 02:30:04.204074 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 08 02:30:04.204074 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f51a5900-5abc-4188-bbc4-33b0f0f7d091 could not be found. Aug 08 02:30:04.204074 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:30:04.204074 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:30:04.204074 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:30:04.204074 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:30:04.204074 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:30:04.204074 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:30:04.204074 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:30:04.204074 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:30:04.204074 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:30:04.208363 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:30:04.208363 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 08 02:30:04.208363 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 08 02:30:04.208363 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 08 02:30:04.208363 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 08 02:30:04.208363 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 08 02:30:04.208363 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:30:04.208363 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f51a5900-5abc-4188-bbc4-33b0f0f7d091 could not be found. Aug 08 02:30:04.208363 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:30:04.208363 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:30:04.208363 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:30:04.208363 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:30:04.208363 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:30:04.208363 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:30:04.208363 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:30:04.208363 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:30:04.208363 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f51a5900-5abc-4188-bbc4-33b0f0f7d091 could not be found. Aug 08 02:30:04.208363 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:30:04.210227 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-b9759340-b614-49fa-9dd2-decbeafb1187 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Returning 404 to user: Instance f51a5900-5abc-4188-bbc4-33b0f0f7d091 could not be found. {{(pid=118352) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:30:04.210227 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-b9759340-b614-49fa-9dd2-decbeafb1187 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 404 len: 111 microversion: 2.20 time: 0.035352 Aug 08 02:30:04.210227 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 300/600] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:30:04 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Aug 08 02:30:04.213458 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.compute.server_external_events [req-2a28f5da-4fc8-40fe-a716-90aaab7b688c req-a57ac382-edc7-4ee0-a79c-b726e1e2910f service nova] Creating event network-changed:45cf1df5-f1d8-432f-8520-f0edf1de75df for instance bd0f42d4-7cae-4736-8c36-fe5fa144908e on hv-cinder-74014 Aug 08 02:30:04.214921 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-ea1345b6-2a56-48c0-9621-e5e8e1cce7be tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '.version_select at 0x7efc74195280>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:04.218637 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [req-2a28f5da-4fc8-40fe-a716-90aaab7b688c req-a57ac382-edc7-4ee0-a79c-b726e1e2910f service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036370 Aug 08 02:30:04.219958 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 301/601] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:30:04 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:30:04.300845 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-ea1345b6-2a56-48c0-9621-e5e8e1cce7be tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1673745916" status: 204 len: 0 microversion: 2.20 time: 0.091146 Aug 08 02:30:04.301458 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 301/602] 10.222.0.27 () {60 vars in 1262 bytes} [Mon Aug 8 02:30:04 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1673745916 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Aug 08 02:30:04.826163 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-3055db82-55fb-486a-a885-7f898f5dd3ff tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:04.826876 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-3055db82-55fb-486a-a885-7f898f5dd3ff tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: bd0f42d4-7cae-4736-8c36-fe5fa144908e] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:04.833259 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3055db82-55fb-486a-a885-7f898f5dd3ff tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:04.833962 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3055db82-55fb-486a-a885-7f898f5dd3ff tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:04.834401 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3055db82-55fb-486a-a885-7f898f5dd3ff tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:04.835101 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3055db82-55fb-486a-a885-7f898f5dd3ff tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:04.835559 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3055db82-55fb-486a-a885-7f898f5dd3ff tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:04.835987 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3055db82-55fb-486a-a885-7f898f5dd3ff tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:05.256610 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-3055db82-55fb-486a-a885-7f898f5dd3ff tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5e3c891f2f3148f4aaba41a14d507cdf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7203c52ae1f946bd9717a554316e4f4c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:05.264184 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-3055db82-55fb-486a-a885-7f898f5dd3ff tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "GET /compute/v2.1/servers/bd0f42d4-7cae-4736-8c36-fe5fa144908e" status: 200 len: 1319 microversion: 2.1 time: 0.443108 Aug 08 02:30:05.264768 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 302/603] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:04 2022] GET /compute/v2.1/servers/bd0f42d4-7cae-4736-8c36-fe5fa144908e => generated 1319 bytes in 444 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:05.535796 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-35dc0977-1684-4891-898b-74dab929f016 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Action: 'create', calling method: .version_select at 0x7efc74077ca0>, body: {"keypair": {"name": "tempest-keypair-902346280", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEXTGKpq695be0S/yyQy5qGeegYNlOyY5z+cyOP1jRdOD5etROJCgPkr14gMI1BQ4kd982flKZ+xdOOMU1JdnkteK+cV7CHWbtjROm9ZuVzBVhJMGwD8lLYJPc6IOx4cUg=="}} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:30:05.552188 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.quota [None req-35dc0977-1684-4891-898b-74dab929f016 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Getting quotas for project 7caa43bf43ad4e84b4b648e9ab33bd7f. Resources: {'key_pairs'} {{(pid=118352) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 08 02:30:05.556145 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.quota [None req-35dc0977-1684-4891-898b-74dab929f016 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Getting quotas for user d451fb2f26ce4f50b010fb2a387c4bf3 and project 7caa43bf43ad4e84b4b648e9ab33bd7f. Resources: {'key_pairs'} {{(pid=118352) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 08 02:30:05.666817 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-35dc0977-1684-4891-898b-74dab929f016 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.20 time: 0.135752 Aug 08 02:30:05.667410 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 302/604] 10.222.0.27 () {62 vars in 1200 bytes} [Mon Aug 8 02:30:05 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 137 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Aug 08 02:30:06.285449 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-f7e9a703-f50c-43e2-b9d3-9325830115a9 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:06.286079 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-f7e9a703-f50c-43e2-b9d3-9325830115a9 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: bd0f42d4-7cae-4736-8c36-fe5fa144908e] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:06.293406 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f7e9a703-f50c-43e2-b9d3-9325830115a9 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:06.293894 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f7e9a703-f50c-43e2-b9d3-9325830115a9 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:06.294329 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f7e9a703-f50c-43e2-b9d3-9325830115a9 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:06.295023 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f7e9a703-f50c-43e2-b9d3-9325830115a9 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:06.295478 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f7e9a703-f50c-43e2-b9d3-9325830115a9 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:06.295900 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f7e9a703-f50c-43e2-b9d3-9325830115a9 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:06.427497 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-f7e9a703-f50c-43e2-b9d3-9325830115a9 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5e3c891f2f3148f4aaba41a14d507cdf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7203c52ae1f946bd9717a554316e4f4c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:06.431074 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:30:06.436110 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-f7e9a703-f50c-43e2-b9d3-9325830115a9 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "GET /compute/v2.1/servers/bd0f42d4-7cae-4736-8c36-fe5fa144908e" status: 200 len: 1319 microversion: 2.1 time: 0.154589 Aug 08 02:30:06.436655 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 303/605] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:06 2022] GET /compute/v2.1/servers/bd0f42d4-7cae-4736-8c36-fe5fa144908e => generated 1319 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:06.550934 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-b6ee26a2-b6a9-49b0-82bd-08e493f5cb2f tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:06.553242 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-b6ee26a2-b6a9-49b0-82bd-08e493f5cb2f tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:06.559478 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b6ee26a2-b6a9-49b0-82bd-08e493f5cb2f tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:06.559940 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b6ee26a2-b6a9-49b0-82bd-08e493f5cb2f tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:06.560375 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b6ee26a2-b6a9-49b0-82bd-08e493f5cb2f tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:06.561092 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b6ee26a2-b6a9-49b0-82bd-08e493f5cb2f tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:06.562898 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b6ee26a2-b6a9-49b0-82bd-08e493f5cb2f tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:06.562898 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b6ee26a2-b6a9-49b0-82bd-08e493f5cb2f tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:06.591477 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-b6ee26a2-b6a9-49b0-82bd-08e493f5cb2f tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.044907 Aug 08 02:30:06.592172 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 303/606] 10.222.0.27 () {58 vars in 1302 bytes} [Mon Aug 8 02:30:06 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a/os-volume_attachments => generated 25 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Aug 08 02:30:06.660572 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-1c083822-c2ec-4b09-b53b-e404641820fe tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:06.661101 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-1c083822-c2ec-4b09-b53b-e404641820fe tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:06.668658 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1c083822-c2ec-4b09-b53b-e404641820fe tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:06.671194 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1c083822-c2ec-4b09-b53b-e404641820fe tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:06.671194 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1c083822-c2ec-4b09-b53b-e404641820fe tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:06.671194 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1c083822-c2ec-4b09-b53b-e404641820fe tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:06.671194 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1c083822-c2ec-4b09-b53b-e404641820fe tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:06.671521 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1c083822-c2ec-4b09-b53b-e404641820fe tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:06.695235 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-1c083822-c2ec-4b09-b53b-e404641820fe tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Going to try to terminate instance {{(pid=118351) delete /opt/stack/nova/nova/compute/api.py:2666}} Aug 08 02:30:07.007485 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-1c083822-c2ec-4b09-b53b-e404641820fe tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "DELETE /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 204 len: 0 microversion: 2.1 time: 0.350445 Aug 08 02:30:07.008313 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 304/607] 10.222.0.27 () {58 vars in 1239 bytes} [Mon Aug 8 02:30:06 2022] DELETE /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 0 bytes in 352 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 08 02:30:07.017244 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-31e3e75f-ecb2-4dfa-8026-8650d0067414 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:07.017859 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-31e3e75f-ecb2-4dfa-8026-8650d0067414 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:07.023689 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-31e3e75f-ecb2-4dfa-8026-8650d0067414 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:07.024173 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-31e3e75f-ecb2-4dfa-8026-8650d0067414 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:07.024606 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-31e3e75f-ecb2-4dfa-8026-8650d0067414 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:07.025351 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-31e3e75f-ecb2-4dfa-8026-8650d0067414 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:07.026356 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-31e3e75f-ecb2-4dfa-8026-8650d0067414 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:07.026782 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-31e3e75f-ecb2-4dfa-8026-8650d0067414 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:07.158056 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-31e3e75f-ecb2-4dfa-8026-8650d0067414 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:07.165728 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-31e3e75f-ecb2-4dfa-8026-8650d0067414 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1676 microversion: 2.1 time: 0.152324 Aug 08 02:30:07.166399 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 304/608] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:07 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1676 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:07.458010 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-e1782465-b62d-41d5-89b1-3ec3b12e10e7 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:07.458790 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-e1782465-b62d-41d5-89b1-3ec3b12e10e7 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: bd0f42d4-7cae-4736-8c36-fe5fa144908e] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:07.465546 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e1782465-b62d-41d5-89b1-3ec3b12e10e7 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:07.466037 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e1782465-b62d-41d5-89b1-3ec3b12e10e7 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:07.466476 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e1782465-b62d-41d5-89b1-3ec3b12e10e7 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:07.467165 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e1782465-b62d-41d5-89b1-3ec3b12e10e7 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:07.467627 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e1782465-b62d-41d5-89b1-3ec3b12e10e7 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:07.468052 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e1782465-b62d-41d5-89b1-3ec3b12e10e7 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:07.614627 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-e1782465-b62d-41d5-89b1-3ec3b12e10e7 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5e3c891f2f3148f4aaba41a14d507cdf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7203c52ae1f946bd9717a554316e4f4c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:07.622895 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-e1782465-b62d-41d5-89b1-3ec3b12e10e7 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "GET /compute/v2.1/servers/bd0f42d4-7cae-4736-8c36-fe5fa144908e" status: 200 len: 1319 microversion: 2.1 time: 0.169208 Aug 08 02:30:07.623473 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 305/609] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:07 2022] GET /compute/v2.1/servers/bd0f42d4-7cae-4736-8c36-fe5fa144908e => generated 1319 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:08.042461 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-3b00e296-2d07-4aa3-9544-521e94b4cced tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-913386058", "imageRef": "30833f47-5987-41ca-b5ee-ad5ecafbb265", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "637774ba-5318-4e0b-8885-18933b06fb53"}], "security_groups": [{"name": "tempest-securitygroup--1509576475"}], "key_name": "tempest-keypair-902346280", "user_data": "}} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:30:08.147158 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.network.neutron [None req-3b00e296-2d07-4aa3-9544-521e94b4cced tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] validate_networks() for [('637774ba-5318-4e0b-8885-18933b06fb53', None, None, None, None, None)] {{(pid=118352) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 08 02:30:08.187161 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-d0a8c999-1143-4c88-adf0-368a82ffa532 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:08.187961 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-d0a8c999-1143-4c88-adf0-368a82ffa532 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:08.194265 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d0a8c999-1143-4c88-adf0-368a82ffa532 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:08.194819 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d0a8c999-1143-4c88-adf0-368a82ffa532 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:08.195262 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d0a8c999-1143-4c88-adf0-368a82ffa532 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:08.195953 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d0a8c999-1143-4c88-adf0-368a82ffa532 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:08.196413 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d0a8c999-1143-4c88-adf0-368a82ffa532 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:08.196902 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d0a8c999-1143-4c88-adf0-368a82ffa532 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:08.395946 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-d0a8c999-1143-4c88-adf0-368a82ffa532 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:08.404828 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-d0a8c999-1143-4c88-adf0-368a82ffa532 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1676 microversion: 2.1 time: 0.221507 Aug 08 02:30:08.405408 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 306/610] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:08 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1676 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:30:08.586355 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-3b00e296-2d07-4aa3-9544-521e94b4cced tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Flavor limits 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 08 02:30:08.586984 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-3b00e296-2d07-4aa3-9544-521e94b4cced tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Image limits 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 08 02:30:08.587690 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-3b00e296-2d07-4aa3-9544-521e94b4cced tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Flavor pref 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 08 02:30:08.588875 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-3b00e296-2d07-4aa3-9544-521e94b4cced tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Image pref 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 08 02:30:08.589463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-3b00e296-2d07-4aa3-9544-521e94b4cced tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 08 02:30:08.600620 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3b00e296-2d07-4aa3-9544-521e94b4cced tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:08.601128 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3b00e296-2d07-4aa3-9544-521e94b4cced tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:08.601562 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3b00e296-2d07-4aa3-9544-521e94b4cced tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:08.602283 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3b00e296-2d07-4aa3-9544-521e94b4cced tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:08.602734 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3b00e296-2d07-4aa3-9544-521e94b4cced tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:08.603157 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3b00e296-2d07-4aa3-9544-521e94b4cced tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:08.619734 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.quota [None req-3b00e296-2d07-4aa3-9544-521e94b4cced tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Getting quotas for project 7caa43bf43ad4e84b4b648e9ab33bd7f. Resources: {'ram', 'cores', 'instances'} {{(pid=118352) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 08 02:30:08.623419 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.quota [None req-3b00e296-2d07-4aa3-9544-521e94b4cced tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Getting quotas for user d451fb2f26ce4f50b010fb2a387c4bf3 and project 7caa43bf43ad4e84b4b648e9ab33bd7f. Resources: {'ram', 'cores', 'instances'} {{(pid=118352) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 08 02:30:08.634162 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-3b00e296-2d07-4aa3-9544-521e94b4cced tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Going to run 1 instances... {{(pid=118352) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Aug 08 02:30:08.636849 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-3b00e296-2d07-4aa3-9544-521e94b4cced tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='30833f47-5987-41ca-b5ee-ad5ecafbb265',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118352) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Aug 08 02:30:08.643361 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-8343e3e8-1328-4037-820f-a6e8cdeea662 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:08.643969 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-8343e3e8-1328-4037-820f-a6e8cdeea662 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: bd0f42d4-7cae-4736-8c36-fe5fa144908e] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:08.651847 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8343e3e8-1328-4037-820f-a6e8cdeea662 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:08.652365 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8343e3e8-1328-4037-820f-a6e8cdeea662 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:08.652871 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8343e3e8-1328-4037-820f-a6e8cdeea662 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:08.653589 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8343e3e8-1328-4037-820f-a6e8cdeea662 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:08.654039 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8343e3e8-1328-4037-820f-a6e8cdeea662 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:08.654457 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8343e3e8-1328-4037-820f-a6e8cdeea662 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:08.764424 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-3b00e296-2d07-4aa3-9544-521e94b4cced tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.20 time: 0.727514 Aug 08 02:30:08.765656 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 305/611] 10.222.0.27 () {62 vars in 1188 bytes} [Mon Aug 8 02:30:08 2022] POST /compute/v2.1/servers => generated 402 bytes in 729 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Aug 08 02:30:08.779374 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-93a94ead-5653-4512-be16-81789f93638c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:08.779990 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-93a94ead-5653-4512-be16-81789f93638c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:08.789378 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-8343e3e8-1328-4037-820f-a6e8cdeea662 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5e3c891f2f3148f4aaba41a14d507cdf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7203c52ae1f946bd9717a554316e4f4c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:08.797788 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-8343e3e8-1328-4037-820f-a6e8cdeea662 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "GET /compute/v2.1/servers/bd0f42d4-7cae-4736-8c36-fe5fa144908e" status: 200 len: 1319 microversion: 2.1 time: 0.158274 Aug 08 02:30:08.798362 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 307/612] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:08 2022] GET /compute/v2.1/servers/bd0f42d4-7cae-4736-8c36-fe5fa144908e => generated 1319 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:30:08.836849 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-93a94ead-5653-4512-be16-81789f93638c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:08.846906 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-93a94ead-5653-4512-be16-81789f93638c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:08.848469 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-93a94ead-5653-4512-be16-81789f93638c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:08.849683 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-93a94ead-5653-4512-be16-81789f93638c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1280 microversion: 2.20 time: 0.075886 Aug 08 02:30:08.851456 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 306/613] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:30:08 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1280 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:30:09.426967 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-db33dcb7-a4c7-4682-9a95-3e1f08515c25 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:09.427743 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-db33dcb7-a4c7-4682-9a95-3e1f08515c25 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:09.434698 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-db33dcb7-a4c7-4682-9a95-3e1f08515c25 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:09.435238 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-db33dcb7-a4c7-4682-9a95-3e1f08515c25 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:09.435702 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-db33dcb7-a4c7-4682-9a95-3e1f08515c25 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:09.436390 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-db33dcb7-a4c7-4682-9a95-3e1f08515c25 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:09.436911 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-db33dcb7-a4c7-4682-9a95-3e1f08515c25 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:09.437333 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-db33dcb7-a4c7-4682-9a95-3e1f08515c25 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:09.593928 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-db33dcb7-a4c7-4682-9a95-3e1f08515c25 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:09.602315 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-db33dcb7-a4c7-4682-9a95-3e1f08515c25 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1676 microversion: 2.1 time: 0.179139 Aug 08 02:30:09.602895 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 308/614] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:09 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1676 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:09.693722 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [req-5f49b217-3525-4a96-ab1d-721b99f01eb1 req-2177fbd6-1d01-464c-a6b1-eaafe690f61f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bd0f42d4-7cae-4736-8c36-fe5fa144908e", "name": "network-vif-plugged", "status": "completed", "tag": "45cf1df5-f1d8-432f-8520-f0edf1de75df"}]} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:30:09.706392 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-5f49b217-3525-4a96-ab1d-721b99f01eb1 req-2177fbd6-1d01-464c-a6b1-eaafe690f61f service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:09.707114 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-5f49b217-3525-4a96-ab1d-721b99f01eb1 req-2177fbd6-1d01-464c-a6b1-eaafe690f61f service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:09.707554 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-5f49b217-3525-4a96-ab1d-721b99f01eb1 req-2177fbd6-1d01-464c-a6b1-eaafe690f61f service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:09.720106 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.compute.server_external_events [req-5f49b217-3525-4a96-ab1d-721b99f01eb1 req-2177fbd6-1d01-464c-a6b1-eaafe690f61f service nova] Creating event network-vif-plugged:45cf1df5-f1d8-432f-8520-f0edf1de75df for instance bd0f42d4-7cae-4736-8c36-fe5fa144908e on hv-cinder-74014 Aug 08 02:30:09.725373 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [req-5f49b217-3525-4a96-ab1d-721b99f01eb1 req-2177fbd6-1d01-464c-a6b1-eaafe690f61f service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035742 Aug 08 02:30:09.725980 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 307/615] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:30:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:30:09.819586 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-f706ec75-d4be-422c-b07b-9e3ac1340835 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:09.820206 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-f706ec75-d4be-422c-b07b-9e3ac1340835 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: bd0f42d4-7cae-4736-8c36-fe5fa144908e] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:09.829083 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f706ec75-d4be-422c-b07b-9e3ac1340835 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:09.830449 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f706ec75-d4be-422c-b07b-9e3ac1340835 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:09.830848 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f706ec75-d4be-422c-b07b-9e3ac1340835 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:09.831779 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f706ec75-d4be-422c-b07b-9e3ac1340835 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:09.832524 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f706ec75-d4be-422c-b07b-9e3ac1340835 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:09.833459 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f706ec75-d4be-422c-b07b-9e3ac1340835 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:09.877459 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-2e300c9c-5cc6-4116-9fdf-fccea9406b85 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:09.878513 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-2e300c9c-5cc6-4116-9fdf-fccea9406b85 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:09.886518 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2e300c9c-5cc6-4116-9fdf-fccea9406b85 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:09.887095 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2e300c9c-5cc6-4116-9fdf-fccea9406b85 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:09.887721 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2e300c9c-5cc6-4116-9fdf-fccea9406b85 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:09.888490 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2e300c9c-5cc6-4116-9fdf-fccea9406b85 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:09.889070 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2e300c9c-5cc6-4116-9fdf-fccea9406b85 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:09.889561 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2e300c9c-5cc6-4116-9fdf-fccea9406b85 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:09.983406 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-2e300c9c-5cc6-4116-9fdf-fccea9406b85 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:09.993150 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-2e300c9c-5cc6-4116-9fdf-fccea9406b85 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:09.994714 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-2e300c9c-5cc6-4116-9fdf-fccea9406b85 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:09.999788 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-2e300c9c-5cc6-4116-9fdf-fccea9406b85 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1284 microversion: 2.20 time: 0.124234 Aug 08 02:30:09.999788 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 308/616] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:30:09 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1284 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:30:10.014284 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-f706ec75-d4be-422c-b07b-9e3ac1340835 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5e3c891f2f3148f4aaba41a14d507cdf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7203c52ae1f946bd9717a554316e4f4c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:10.023955 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-f706ec75-d4be-422c-b07b-9e3ac1340835 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "GET /compute/v2.1/servers/bd0f42d4-7cae-4736-8c36-fe5fa144908e" status: 200 len: 1319 microversion: 2.1 time: 0.208478 Aug 08 02:30:10.028578 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 309/617] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:09 2022] GET /compute/v2.1/servers/bd0f42d4-7cae-4736-8c36-fe5fa144908e => generated 1319 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:10.635039 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-98b8ee9e-f219-48c9-ba9a-14ef3e4c6192 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:10.637649 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-98b8ee9e-f219-48c9-ba9a-14ef3e4c6192 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:10.646317 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-98b8ee9e-f219-48c9-ba9a-14ef3e4c6192 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:10.650441 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-98b8ee9e-f219-48c9-ba9a-14ef3e4c6192 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:10.650441 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-98b8ee9e-f219-48c9-ba9a-14ef3e4c6192 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:10.650441 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-98b8ee9e-f219-48c9-ba9a-14ef3e4c6192 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:10.650441 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-98b8ee9e-f219-48c9-ba9a-14ef3e4c6192 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:10.651442 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-98b8ee9e-f219-48c9-ba9a-14ef3e4c6192 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:10.739798 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-98b8ee9e-f219-48c9-ba9a-14ef3e4c6192 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:10.749739 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-98b8ee9e-f219-48c9-ba9a-14ef3e4c6192 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1608 microversion: 2.1 time: 0.124400 Aug 08 02:30:10.750927 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 309/618] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:10 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1608 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:10.845168 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [req-c5a2b141-38a2-45de-9dc9-29e59fe6c67e req-85f95180-8def-40b9-8fd2-0b8ee087e47e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ddd15bdd-8a1c-455b-91c9-504f69b5308a", "name": "network-vif-deleted", "tag": "84fa14fb-cbbd-42d5-a597-bc2556a0ba82"}]} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:30:10.850505 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell0:SELECT=2 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:30:10.858046 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-c5a2b141-38a2-45de-9dc9-29e59fe6c67e req-85f95180-8def-40b9-8fd2-0b8ee087e47e service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:10.858622 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-c5a2b141-38a2-45de-9dc9-29e59fe6c67e req-85f95180-8def-40b9-8fd2-0b8ee087e47e service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:10.859055 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-c5a2b141-38a2-45de-9dc9-29e59fe6c67e req-85f95180-8def-40b9-8fd2-0b8ee087e47e service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:10.871554 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.compute.server_external_events [req-c5a2b141-38a2-45de-9dc9-29e59fe6c67e req-85f95180-8def-40b9-8fd2-0b8ee087e47e service nova] Creating event network-vif-deleted:84fa14fb-cbbd-42d5-a597-bc2556a0ba82 for instance ddd15bdd-8a1c-455b-91c9-504f69b5308a on hv-cinder-74014 Aug 08 02:30:10.876745 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [req-c5a2b141-38a2-45de-9dc9-29e59fe6c67e req-85f95180-8def-40b9-8fd2-0b8ee087e47e service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035887 Aug 08 02:30:10.877847 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 310/619] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:30:10 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:30:11.032457 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-aadc9904-440c-4eb1-8ac8-4c214c7a0468 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:11.033085 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-aadc9904-440c-4eb1-8ac8-4c214c7a0468 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:11.039417 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-aadc9904-440c-4eb1-8ac8-4c214c7a0468 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:11.042841 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-aadc9904-440c-4eb1-8ac8-4c214c7a0468 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:11.042841 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-aadc9904-440c-4eb1-8ac8-4c214c7a0468 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:11.042841 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-aadc9904-440c-4eb1-8ac8-4c214c7a0468 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:11.042841 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-aadc9904-440c-4eb1-8ac8-4c214c7a0468 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:11.043208 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-aadc9904-440c-4eb1-8ac8-4c214c7a0468 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:11.052480 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-01637d1a-b499-4bb3-abf9-88e107ed3da6 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:11.053228 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-01637d1a-b499-4bb3-abf9-88e107ed3da6 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: bd0f42d4-7cae-4736-8c36-fe5fa144908e] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:11.061648 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-01637d1a-b499-4bb3-abf9-88e107ed3da6 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:11.065672 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-01637d1a-b499-4bb3-abf9-88e107ed3da6 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:11.065672 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-01637d1a-b499-4bb3-abf9-88e107ed3da6 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:11.065672 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-01637d1a-b499-4bb3-abf9-88e107ed3da6 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:11.065672 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-01637d1a-b499-4bb3-abf9-88e107ed3da6 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:11.066074 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-01637d1a-b499-4bb3-abf9-88e107ed3da6 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:11.117200 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-aadc9904-440c-4eb1-8ac8-4c214c7a0468 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:11.127509 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-aadc9904-440c-4eb1-8ac8-4c214c7a0468 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:11.129292 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-aadc9904-440c-4eb1-8ac8-4c214c7a0468 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:11.130484 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-aadc9904-440c-4eb1-8ac8-4c214c7a0468 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1332 microversion: 2.20 time: 0.102013 Aug 08 02:30:11.131155 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 310/620] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:30:11 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1332 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:30:11.221141 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-01637d1a-b499-4bb3-abf9-88e107ed3da6 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5e3c891f2f3148f4aaba41a14d507cdf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7203c52ae1f946bd9717a554316e4f4c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:11.228623 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-01637d1a-b499-4bb3-abf9-88e107ed3da6 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "GET /compute/v2.1/servers/bd0f42d4-7cae-4736-8c36-fe5fa144908e" status: 200 len: 1319 microversion: 2.1 time: 0.182407 Aug 08 02:30:11.229891 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 311/621] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:11 2022] GET /compute/v2.1/servers/bd0f42d4-7cae-4736-8c36-fe5fa144908e => generated 1319 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:11.770586 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-23010ea3-5b54-47d0-bee2-fb88439b1343 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:11.771276 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-23010ea3-5b54-47d0-bee2-fb88439b1343 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:11.778728 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-23010ea3-5b54-47d0-bee2-fb88439b1343 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:11.781147 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-23010ea3-5b54-47d0-bee2-fb88439b1343 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:11.781147 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-23010ea3-5b54-47d0-bee2-fb88439b1343 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:11.781147 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-23010ea3-5b54-47d0-bee2-fb88439b1343 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:11.781147 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-23010ea3-5b54-47d0-bee2-fb88439b1343 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:11.781596 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-23010ea3-5b54-47d0-bee2-fb88439b1343 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:11.814395 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_api:SELECT=92,nova_api:INSERT=7,nova_api:DELETE=1 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:30:11.818561 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-23010ea3-5b54-47d0-bee2-fb88439b1343 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:11.819047 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-23010ea3-5b54-47d0-bee2-fb88439b1343 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:11.819486 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-23010ea3-5b54-47d0-bee2-fb88439b1343 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:11.819929 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.objects.instance [None req-23010ea3-5b54-47d0-bee2-fb88439b1343 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lazy-loading 'fault' on Instance uuid ddd15bdd-8a1c-455b-91c9-504f69b5308a {{(pid=118352) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 08 02:30:11.863050 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-23010ea3-5b54-47d0-bee2-fb88439b1343 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:11.867793 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-23010ea3-5b54-47d0-bee2-fb88439b1343 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 200 len: 1328 microversion: 2.1 time: 0.101059 Aug 08 02:30:11.868358 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 311/622] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:11 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 1328 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:12.155920 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-e4b47731-c9e6-4815-bc5d-483523960292 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:12.156528 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-e4b47731-c9e6-4815-bc5d-483523960292 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:12.162914 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e4b47731-c9e6-4815-bc5d-483523960292 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:12.163396 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e4b47731-c9e6-4815-bc5d-483523960292 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:12.163832 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e4b47731-c9e6-4815-bc5d-483523960292 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:12.164564 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e4b47731-c9e6-4815-bc5d-483523960292 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:12.165050 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e4b47731-c9e6-4815-bc5d-483523960292 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:12.165473 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e4b47731-c9e6-4815-bc5d-483523960292 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:12.229903 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-e4b47731-c9e6-4815-bc5d-483523960292 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:12.238226 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-e4b47731-c9e6-4815-bc5d-483523960292 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:12.239789 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-e4b47731-c9e6-4815-bc5d-483523960292 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:12.240899 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-e4b47731-c9e6-4815-bc5d-483523960292 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1340 microversion: 2.20 time: 0.089085 Aug 08 02:30:12.241461 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 312/623] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:30:12 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1340 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:30:12.250476 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-ded0273b-6ebf-40f2-b6b0-8a49489acee8 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:12.251178 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-ded0273b-6ebf-40f2-b6b0-8a49489acee8 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: bd0f42d4-7cae-4736-8c36-fe5fa144908e] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:12.257509 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ded0273b-6ebf-40f2-b6b0-8a49489acee8 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:12.257992 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ded0273b-6ebf-40f2-b6b0-8a49489acee8 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:12.258421 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ded0273b-6ebf-40f2-b6b0-8a49489acee8 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:12.259120 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ded0273b-6ebf-40f2-b6b0-8a49489acee8 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:12.259569 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ded0273b-6ebf-40f2-b6b0-8a49489acee8 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:12.259995 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ded0273b-6ebf-40f2-b6b0-8a49489acee8 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:12.421384 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-ded0273b-6ebf-40f2-b6b0-8a49489acee8 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5e3c891f2f3148f4aaba41a14d507cdf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7203c52ae1f946bd9717a554316e4f4c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:12.430148 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-ded0273b-6ebf-40f2-b6b0-8a49489acee8 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "GET /compute/v2.1/servers/bd0f42d4-7cae-4736-8c36-fe5fa144908e" status: 200 len: 1508 microversion: 2.1 time: 0.183833 Aug 08 02:30:12.438585 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 312/624] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:12 2022] GET /compute/v2.1/servers/bd0f42d4-7cae-4736-8c36-fe5fa144908e => generated 1508 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:12.451475 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-f80edff1-1144-48d4-af71-e2a2ba617398 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "17ffdefc-f349-47c8-b130-6d758fe1db67", "device": "/dev/sdb"}} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:30:12.454181 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-f80edff1-1144-48d4-af71-e2a2ba617398 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: bd0f42d4-7cae-4736-8c36-fe5fa144908e] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:12.460379 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f80edff1-1144-48d4-af71-e2a2ba617398 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:12.460904 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f80edff1-1144-48d4-af71-e2a2ba617398 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:12.461338 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f80edff1-1144-48d4-af71-e2a2ba617398 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:12.462052 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f80edff1-1144-48d4-af71-e2a2ba617398 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:12.462547 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f80edff1-1144-48d4-af71-e2a2ba617398 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:12.463165 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f80edff1-1144-48d4-af71-e2a2ba617398 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:12.889424 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-b25f3dd4-741b-4961-a65a-91da2b40a70d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:12.890663 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-b25f3dd4-741b-4961-a65a-91da2b40a70d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:12.897050 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b25f3dd4-741b-4961-a65a-91da2b40a70d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:12.897743 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b25f3dd4-741b-4961-a65a-91da2b40a70d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:12.898175 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b25f3dd4-741b-4961-a65a-91da2b40a70d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:12.898871 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b25f3dd4-741b-4961-a65a-91da2b40a70d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:12.899326 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b25f3dd4-741b-4961-a65a-91da2b40a70d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:12.899823 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b25f3dd4-741b-4961-a65a-91da2b40a70d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:12.915384 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.wsgi [None req-b25f3dd4-741b-4961-a65a-91da2b40a70d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] HTTP exception thrown: Instance ddd15bdd-8a1c-455b-91c9-504f69b5308a could not be found. Aug 08 02:30:12.918370 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-b25f3dd4-741b-4961-a65a-91da2b40a70d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Request method failure captured: Aug 08 02:30:12.918370 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: request: GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a HTTP/1.1 Aug 08 02:30:12.918370 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept: application/json Aug 08 02:30:12.918370 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept-Encoding: identity Aug 08 02:30:12.918370 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Connection: close Aug 08 02:30:12.918370 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Length: 0 Aug 08 02:30:12.918370 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Type: application/json Aug 08 02:30:12.918370 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Host: 10.222.0.27 Aug 08 02:30:12.918370 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Openstack-System-Scope: None Aug 08 02:30:12.918370 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: User-Agent: python-urllib3/1.26.11 Aug 08 02:30:12.918370 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Auth-Token: gAAAAABi8HQalNSG8U0tawdT8BYmZ3dO_Z_XJ7b_zGy3jhslHecNN8lPHCadrD9L1wE_fNDeX2KYU182-SQacEPppcF5Uwunh8fURGrax-nwHUph2qRhEJoRudpg1Feo8sNMetQ07fI55mt7EEM91lToca7t2N_CdvJ1rSwFvtzAdQ4uUEvan-8 Aug 08 02:30:12.918370 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Id: None Aug 08 02:30:12.918370 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Name: None Aug 08 02:30:12.918370 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Identity-Status: Confirmed Aug 08 02:30:12.918370 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Is-Admin-Project: True Aug 08 02:30:12.918370 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Id: default Aug 08 02:30:12.918370 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Name: Default Aug 08 02:30:12.918370 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Id: ee8baa7d6e294e27ae2fb12eded3709d Aug 08 02:30:12.918370 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Name: tempest-AttachVolumeTestJSON-827912071 Aug 08 02:30:12.918370 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Role: reader,member Aug 08 02:30:12.918370 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Roles: reader,member Aug 08 02:30:12.925693 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/ee8baa7d6e294e27ae2fb12eded3709d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/ee8baa7d6e294e27ae2fb12eded3709d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_ee8baa7d6e294e27ae2fb12eded3709d", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/ee8baa7d6e294e27ae2fb12eded3709d"}]}] Aug 08 02:30:12.925693 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant: tempest-AttachVolumeTestJSON-827912071 Aug 08 02:30:12.925693 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Id: ee8baa7d6e294e27ae2fb12eded3709d Aug 08 02:30:12.925693 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Name: tempest-AttachVolumeTestJSON-827912071 Aug 08 02:30:12.925693 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User: tempest-AttachVolumeTestJSON-827912071-project Aug 08 02:30:12.925693 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Id: default Aug 08 02:30:12.925693 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Name: Default Aug 08 02:30:12.925693 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Id: eb4c92edcb024e32a549650eb85f6952 Aug 08 02:30:12.925693 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Name: tempest-AttachVolumeTestJSON-827912071-project Aug 08 02:30:12.925693 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: method: > Aug 08 02:30:12.925693 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: exception: Instance ddd15bdd-8a1c-455b-91c9-504f69b5308a could not be found. Aug 08 02:30:12.925693 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: action_args: {'id': 'ddd15bdd-8a1c-455b-91c9-504f69b5308a'} Aug 08 02:30:12.926463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:30:12.926463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:30:12.926463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 08 02:30:12.926463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 08 02:30:12.926463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 08 02:30:12.926463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 08 02:30:12.926463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 08 02:30:12.926463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 08 02:30:12.926463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 08 02:30:12.926463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 08 02:30:12.926463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ddd15bdd-8a1c-455b-91c9-504f69b5308a could not be found. Aug 08 02:30:12.926463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:30:12.926463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:30:12.926463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:30:12.926463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:30:12.926463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:30:12.926463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:30:12.926463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:30:12.926463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:30:12.926463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:30:12.927695 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:30:12.927695 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 08 02:30:12.927695 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 08 02:30:12.927695 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 08 02:30:12.927695 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 08 02:30:12.927695 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 08 02:30:12.927695 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:30:12.927695 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ddd15bdd-8a1c-455b-91c9-504f69b5308a could not be found. Aug 08 02:30:12.927695 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:30:12.927695 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:30:12.927695 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:30:12.927695 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:30:12.927695 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:30:12.927695 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:30:12.927695 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:30:12.927695 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:30:12.927695 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ddd15bdd-8a1c-455b-91c9-504f69b5308a could not be found. Aug 08 02:30:12.927695 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:30:12.928904 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-b25f3dd4-741b-4961-a65a-91da2b40a70d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Returning 404 to user: Instance ddd15bdd-8a1c-455b-91c9-504f69b5308a could not be found. {{(pid=118352) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:30:12.928904 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-b25f3dd4-741b-4961-a65a-91da2b40a70d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 404 len: 111 microversion: 2.1 time: 0.035407 Aug 08 02:30:12.928904 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 313/625] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:12 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Aug 08 02:30:12.928904 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-69e1c4f1-ebb9-4e9e-a21b-05b58a3a2186 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '.version_select at 0x7efc7431cee0>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:12.998598 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-69e1c4f1-ebb9-4e9e-a21b-05b58a3a2186 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-2085665456" status: 202 len: 0 microversion: 2.1 time: 0.073735 Aug 08 02:30:12.999202 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 314/626] 10.222.0.27 () {58 vars in 1221 bytes} [Mon Aug 8 02:30:12 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-2085665456 => generated 0 bytes in 75 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Aug 08 02:30:13.114205 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-8938f775-15ff-4c17-ade8-dd10afecfc93 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:13.115068 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-8938f775-15ff-4c17-ade8-dd10afecfc93 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:13.123634 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8938f775-15ff-4c17-ade8-dd10afecfc93 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:13.126974 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8938f775-15ff-4c17-ade8-dd10afecfc93 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:13.126974 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8938f775-15ff-4c17-ade8-dd10afecfc93 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:13.126974 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8938f775-15ff-4c17-ade8-dd10afecfc93 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:13.126974 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8938f775-15ff-4c17-ade8-dd10afecfc93 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:13.127397 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8938f775-15ff-4c17-ade8-dd10afecfc93 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:13.215361 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.objects.instance [None req-8938f775-15ff-4c17-ade8-dd10afecfc93 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lazy-loading 'services' on Instance uuid a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 {{(pid=118352) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 08 02:30:13.341126 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-f80edff1-1144-48d4-af71-e2a2ba617398 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "POST /compute/v2.1/servers/bd0f42d4-7cae-4736-8c36-fe5fa144908e/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.892977 Aug 08 02:30:13.342418 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 313/627] 10.222.0.27 () {60 vars in 1323 bytes} [Mon Aug 8 02:30:12 2022] POST /compute/v2.1/servers/bd0f42d4-7cae-4736-8c36-fe5fa144908e/os-volume_attachments => generated 194 bytes in 896 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:30:13.365225 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-f7177916-b42d-46fa-87f6-1078e83048b6 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:13.366284 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-f7177916-b42d-46fa-87f6-1078e83048b6 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:13.374729 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f7177916-b42d-46fa-87f6-1078e83048b6 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:13.375634 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f7177916-b42d-46fa-87f6-1078e83048b6 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:13.376366 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f7177916-b42d-46fa-87f6-1078e83048b6 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:13.377220 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f7177916-b42d-46fa-87f6-1078e83048b6 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:13.378064 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f7177916-b42d-46fa-87f6-1078e83048b6 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:13.378592 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f7177916-b42d-46fa-87f6-1078e83048b6 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:13.390251 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:30:13.512948 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-8938f775-15ff-4c17-ade8-dd10afecfc93 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "DELETE /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24/os-volume_attachments/7cdab395-b182-49c3-8bf5-3e6673f0dc52" status: 202 len: 0 microversion: 2.1 time: 0.402414 Aug 08 02:30:13.513984 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 315/628] 10.222.0.27 () {58 vars in 1416 bytes} [Mon Aug 8 02:30:13 2022] DELETE /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24/os-volume_attachments/7cdab395-b182-49c3-8bf5-3e6673f0dc52 => generated 0 bytes in 404 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Aug 08 02:30:13.533009 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-9ff5cc1e-50fe-4853-bbaa-e9e0a0ac5011 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:13.538191 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-9ff5cc1e-50fe-4853-bbaa-e9e0a0ac5011 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:13.550875 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9ff5cc1e-50fe-4853-bbaa-e9e0a0ac5011 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:13.551840 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9ff5cc1e-50fe-4853-bbaa-e9e0a0ac5011 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:13.552342 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9ff5cc1e-50fe-4853-bbaa-e9e0a0ac5011 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:13.553547 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9ff5cc1e-50fe-4853-bbaa-e9e0a0ac5011 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:13.554488 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9ff5cc1e-50fe-4853-bbaa-e9e0a0ac5011 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:13.556058 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9ff5cc1e-50fe-4853-bbaa-e9e0a0ac5011 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:13.585817 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-f7177916-b42d-46fa-87f6-1078e83048b6 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:13.605220 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-f7177916-b42d-46fa-87f6-1078e83048b6 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:13.607633 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-f7177916-b42d-46fa-87f6-1078e83048b6 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:13.610349 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-f7177916-b42d-46fa-87f6-1078e83048b6 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1406 microversion: 2.20 time: 0.266422 Aug 08 02:30:13.611501 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 314/629] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:30:13 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1406 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:30:13.616028 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-9ff5cc1e-50fe-4853-bbaa-e9e0a0ac5011 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.094671 Aug 08 02:30:13.616639 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 316/630] 10.222.0.27 () {58 vars in 1302 bytes} [Mon Aug 8 02:30:13 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24/os-volume_attachments => generated 197 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:30:14.241067 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-a15e9c70-0fc3-4d3a-a5d8-543ac40f899e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Action: 'create', calling method: .version_select at 0x7efc74158f70>, body: {"keypair": {"name": "tempest-keypair-2085281366", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPrcJBy8Za5CwFzJ20PV0oDYq4ZmfUvzdLXY9+XapT+UT3c98vRq3IAcNPdlmapC85a5EcHpKC7+BNOVqnJ57OldE9RNf5cSx74i8peJEua7+QyNZ1NJncH8Wn+IoaQpFg=="}} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:30:14.258298 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.quota [None req-a15e9c70-0fc3-4d3a-a5d8-543ac40f899e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Getting quotas for project ee8baa7d6e294e27ae2fb12eded3709d. Resources: {'key_pairs'} {{(pid=118351) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 08 02:30:14.262966 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.quota [None req-a15e9c70-0fc3-4d3a-a5d8-543ac40f899e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Getting quotas for user eb4c92edcb024e32a549650eb85f6952 and project ee8baa7d6e294e27ae2fb12eded3709d. Resources: {'key_pairs'} {{(pid=118351) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 08 02:30:14.364082 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-a15e9c70-0fc3-4d3a-a5d8-543ac40f899e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.129528 Aug 08 02:30:14.364843 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 315/631] 10.222.0.27 () {60 vars in 1159 bytes} [Mon Aug 8 02:30:14 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:30:14.453660 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [req-2d123bb6-27a2-4852-abac-e546e7f4858b req-dfbf265b-8d26-4505-826b-975a6d177aeb service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "946fd571-bd98-44f9-89f4-52f0116b642d", "tag": "0bf885ea-ca94-45b2-aafa-7461404e6cf2"}]} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:30:14.466123 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-2d123bb6-27a2-4852-abac-e546e7f4858b req-dfbf265b-8d26-4505-826b-975a6d177aeb service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:14.472638 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-2d123bb6-27a2-4852-abac-e546e7f4858b req-dfbf265b-8d26-4505-826b-975a6d177aeb service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:14.472638 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-2d123bb6-27a2-4852-abac-e546e7f4858b req-dfbf265b-8d26-4505-826b-975a6d177aeb service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:14.491132 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.compute.server_external_events [req-2d123bb6-27a2-4852-abac-e546e7f4858b req-dfbf265b-8d26-4505-826b-975a6d177aeb service nova] Creating event network-changed:0bf885ea-ca94-45b2-aafa-7461404e6cf2 for instance 946fd571-bd98-44f9-89f4-52f0116b642d on hv-cinder-74014 Aug 08 02:30:14.497398 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [req-2d123bb6-27a2-4852-abac-e546e7f4858b req-dfbf265b-8d26-4505-826b-975a6d177aeb service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.047205 Aug 08 02:30:14.498201 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 317/632] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:30:14 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:30:14.628359 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-2e8a3ddd-5732-4f74-9291-82aa4f034afa tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:14.630750 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-2e8a3ddd-5732-4f74-9291-82aa4f034afa tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:14.635304 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-7bcc4fb1-73f8-4e6d-8a54-5b1e62284a83 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:14.636446 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-7bcc4fb1-73f8-4e6d-8a54-5b1e62284a83 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:14.638532 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2e8a3ddd-5732-4f74-9291-82aa4f034afa tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:14.639100 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2e8a3ddd-5732-4f74-9291-82aa4f034afa tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:14.639632 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2e8a3ddd-5732-4f74-9291-82aa4f034afa tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:14.640574 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2e8a3ddd-5732-4f74-9291-82aa4f034afa tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:14.641300 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2e8a3ddd-5732-4f74-9291-82aa4f034afa tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:14.641729 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2e8a3ddd-5732-4f74-9291-82aa4f034afa tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:14.643539 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7bcc4fb1-73f8-4e6d-8a54-5b1e62284a83 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:14.644032 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7bcc4fb1-73f8-4e6d-8a54-5b1e62284a83 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:14.644497 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7bcc4fb1-73f8-4e6d-8a54-5b1e62284a83 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:14.645330 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7bcc4fb1-73f8-4e6d-8a54-5b1e62284a83 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:14.646772 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7bcc4fb1-73f8-4e6d-8a54-5b1e62284a83 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:14.646974 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7bcc4fb1-73f8-4e6d-8a54-5b1e62284a83 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:14.674532 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-2e8a3ddd-5732-4f74-9291-82aa4f034afa tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050525 Aug 08 02:30:14.675636 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 316/633] 10.222.0.27 () {58 vars in 1302 bytes} [Mon Aug 8 02:30:14 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:30:14.785502 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-7bcc4fb1-73f8-4e6d-8a54-5b1e62284a83 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:14.794280 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-7bcc4fb1-73f8-4e6d-8a54-5b1e62284a83 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:14.796216 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-7bcc4fb1-73f8-4e6d-8a54-5b1e62284a83 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:14.797367 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-7bcc4fb1-73f8-4e6d-8a54-5b1e62284a83 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1406 microversion: 2.20 time: 0.166623 Aug 08 02:30:14.807990 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 318/634] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:30:14 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1406 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 08 02:30:15.687689 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-dc7ec8b4-5707-4e8b-9e2a-5f09dc9ffdf0 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:15.689921 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-dc7ec8b4-5707-4e8b-9e2a-5f09dc9ffdf0 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:15.696677 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-dc7ec8b4-5707-4e8b-9e2a-5f09dc9ffdf0 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:15.697176 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-dc7ec8b4-5707-4e8b-9e2a-5f09dc9ffdf0 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:15.698371 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-dc7ec8b4-5707-4e8b-9e2a-5f09dc9ffdf0 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:15.699053 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-dc7ec8b4-5707-4e8b-9e2a-5f09dc9ffdf0 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:15.699509 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-dc7ec8b4-5707-4e8b-9e2a-5f09dc9ffdf0 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:15.699932 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-dc7ec8b4-5707-4e8b-9e2a-5f09dc9ffdf0 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:15.737988 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-dc7ec8b4-5707-4e8b-9e2a-5f09dc9ffdf0 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054459 Aug 08 02:30:15.738941 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 317/635] 10.222.0.27 () {58 vars in 1302 bytes} [Mon Aug 8 02:30:15 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:30:15.820498 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-12c926c9-9256-48f4-98d6-199e92ba7ddf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:15.821537 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-12c926c9-9256-48f4-98d6-199e92ba7ddf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:15.829458 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-12c926c9-9256-48f4-98d6-199e92ba7ddf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:15.830170 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-12c926c9-9256-48f4-98d6-199e92ba7ddf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:15.830759 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-12c926c9-9256-48f4-98d6-199e92ba7ddf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:15.831464 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-12c926c9-9256-48f4-98d6-199e92ba7ddf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:15.832085 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-12c926c9-9256-48f4-98d6-199e92ba7ddf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:15.832513 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-12c926c9-9256-48f4-98d6-199e92ba7ddf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:16.041351 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-12c926c9-9256-48f4-98d6-199e92ba7ddf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:16.051862 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-12c926c9-9256-48f4-98d6-199e92ba7ddf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:16.053408 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-12c926c9-9256-48f4-98d6-199e92ba7ddf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:16.054661 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-12c926c9-9256-48f4-98d6-199e92ba7ddf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1406 microversion: 2.20 time: 0.238225 Aug 08 02:30:16.055223 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 319/636] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:30:15 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1406 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:30:16.750490 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-97f7972e-c811-4944-ac37-91cd2ca519b0 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:16.752613 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-97f7972e-c811-4944-ac37-91cd2ca519b0 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:16.759356 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-97f7972e-c811-4944-ac37-91cd2ca519b0 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:16.759865 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-97f7972e-c811-4944-ac37-91cd2ca519b0 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:16.761032 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-97f7972e-c811-4944-ac37-91cd2ca519b0 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:16.761872 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-97f7972e-c811-4944-ac37-91cd2ca519b0 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:16.762330 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-97f7972e-c811-4944-ac37-91cd2ca519b0 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:16.762948 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-97f7972e-c811-4944-ac37-91cd2ca519b0 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:16.795256 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-97f7972e-c811-4944-ac37-91cd2ca519b0 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048739 Aug 08 02:30:16.795829 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 318/637] 10.222.0.27 () {58 vars in 1302 bytes} [Mon Aug 8 02:30:16 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:30:17.019728 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-1062f8df-2c49-4c43-b787-a10602ee227a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-1581784440", "imageRef": "30833f47-5987-41ca-b5ee-ad5ecafbb265", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "23dedbcc-b8b7-4ca6-a0c0-944d2accdccd"}], "security_groups": [{"name": "tempest-securitygroup--2084159102"}], "key_name": "tempest-keypair-2085281366", "user_data": "}} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:30:17.077841 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-28e7bd9f-0f33-40e1-82e4-d8589ecee21b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:17.078534 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-28e7bd9f-0f33-40e1-82e4-d8589ecee21b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:17.084537 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-28e7bd9f-0f33-40e1-82e4-d8589ecee21b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:17.085261 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-28e7bd9f-0f33-40e1-82e4-d8589ecee21b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:17.085728 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-28e7bd9f-0f33-40e1-82e4-d8589ecee21b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:17.086425 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-28e7bd9f-0f33-40e1-82e4-d8589ecee21b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:17.086877 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-28e7bd9f-0f33-40e1-82e4-d8589ecee21b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:17.087302 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-28e7bd9f-0f33-40e1-82e4-d8589ecee21b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:17.091769 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:30:17.136535 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.network.neutron [None req-1062f8df-2c49-4c43-b787-a10602ee227a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] validate_networks() for [('23dedbcc-b8b7-4ca6-a0c0-944d2accdccd', None, None, None, None, None)] {{(pid=118352) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 08 02:30:17.428363 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-28e7bd9f-0f33-40e1-82e4-d8589ecee21b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:17.437870 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-28e7bd9f-0f33-40e1-82e4-d8589ecee21b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:17.439775 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-28e7bd9f-0f33-40e1-82e4-d8589ecee21b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:17.441137 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-28e7bd9f-0f33-40e1-82e4-d8589ecee21b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1406 microversion: 2.20 time: 0.367050 Aug 08 02:30:17.441925 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 319/638] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:30:17 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1406 bytes in 369 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 08 02:30:17.566459 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-1062f8df-2c49-4c43-b787-a10602ee227a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Flavor limits 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 08 02:30:17.566877 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-1062f8df-2c49-4c43-b787-a10602ee227a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Image limits 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 08 02:30:17.567351 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-1062f8df-2c49-4c43-b787-a10602ee227a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Flavor pref 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 08 02:30:17.567790 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-1062f8df-2c49-4c43-b787-a10602ee227a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Image pref 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 08 02:30:17.568193 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-1062f8df-2c49-4c43-b787-a10602ee227a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 08 02:30:17.577945 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1062f8df-2c49-4c43-b787-a10602ee227a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:17.578426 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1062f8df-2c49-4c43-b787-a10602ee227a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:17.578898 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1062f8df-2c49-4c43-b787-a10602ee227a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:17.579619 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1062f8df-2c49-4c43-b787-a10602ee227a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:17.580055 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1062f8df-2c49-4c43-b787-a10602ee227a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:17.580475 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1062f8df-2c49-4c43-b787-a10602ee227a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:17.598727 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.quota [None req-1062f8df-2c49-4c43-b787-a10602ee227a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Getting quotas for project ee8baa7d6e294e27ae2fb12eded3709d. Resources: {'ram', 'cores', 'instances'} {{(pid=118352) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 08 02:30:17.602694 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.quota [None req-1062f8df-2c49-4c43-b787-a10602ee227a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Getting quotas for user eb4c92edcb024e32a549650eb85f6952 and project ee8baa7d6e294e27ae2fb12eded3709d. Resources: {'ram', 'cores', 'instances'} {{(pid=118352) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 08 02:30:17.613789 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-1062f8df-2c49-4c43-b787-a10602ee227a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Going to run 1 instances... {{(pid=118352) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Aug 08 02:30:17.616044 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-1062f8df-2c49-4c43-b787-a10602ee227a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: 48b3d4bf-6b20-4327-8f8d-521417405254] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='30833f47-5987-41ca-b5ee-ad5ecafbb265',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118352) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Aug 08 02:30:17.782272 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-1062f8df-2c49-4c43-b787-a10602ee227a tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.766605 Aug 08 02:30:17.782668 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 320/639] 10.222.0.27 () {60 vars in 1147 bytes} [Mon Aug 8 02:30:17 2022] POST /compute/v2.1/servers => generated 402 bytes in 768 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 08 02:30:17.793627 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-d6fbb9ed-cf7c-415e-bde0-94418a7e2f6c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:17.794578 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-d6fbb9ed-cf7c-415e-bde0-94418a7e2f6c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: 48b3d4bf-6b20-4327-8f8d-521417405254] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:17.811815 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-74e57f46-befe-4fab-8623-e08fff1ee531 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:17.811815 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-74e57f46-befe-4fab-8623-e08fff1ee531 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:17.817811 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-74e57f46-befe-4fab-8623-e08fff1ee531 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:17.818338 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-74e57f46-befe-4fab-8623-e08fff1ee531 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:17.819069 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-74e57f46-befe-4fab-8623-e08fff1ee531 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:17.819773 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-74e57f46-befe-4fab-8623-e08fff1ee531 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:17.820358 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-74e57f46-befe-4fab-8623-e08fff1ee531 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:17.820997 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-74e57f46-befe-4fab-8623-e08fff1ee531 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:17.855917 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-74e57f46-befe-4fab-8623-e08fff1ee531 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052472 Aug 08 02:30:17.860325 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 321/640] 10.222.0.27 () {58 vars in 1302 bytes} [Mon Aug 8 02:30:17 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 08 02:30:17.877610 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-d6fbb9ed-cf7c-415e-bde0-94418a7e2f6c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:17.886080 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-d6fbb9ed-cf7c-415e-bde0-94418a7e2f6c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254" status: 200 len: 1238 microversion: 2.1 time: 0.095748 Aug 08 02:30:17.886644 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 320/641] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:17 2022] GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254 => generated 1238 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:18.464929 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-3f348626-ff9e-42de-8493-047e17228aea tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:18.465541 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-3f348626-ff9e-42de-8493-047e17228aea tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:18.472019 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3f348626-ff9e-42de-8493-047e17228aea tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:18.472501 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3f348626-ff9e-42de-8493-047e17228aea tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:18.472973 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3f348626-ff9e-42de-8493-047e17228aea tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:18.473661 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3f348626-ff9e-42de-8493-047e17228aea tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:18.474114 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3f348626-ff9e-42de-8493-047e17228aea tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:18.474542 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3f348626-ff9e-42de-8493-047e17228aea tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:18.608816 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-3f348626-ff9e-42de-8493-047e17228aea tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:18.617070 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-3f348626-ff9e-42de-8493-047e17228aea tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:18.618580 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-3f348626-ff9e-42de-8493-047e17228aea tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:18.619874 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-3f348626-ff9e-42de-8493-047e17228aea tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1406 microversion: 2.20 time: 0.158959 Aug 08 02:30:18.620455 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 322/642] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:30:18 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1406 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:30:18.868351 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-d8306486-9d95-49ba-be21-d6edca0b7f31 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:18.870655 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-d8306486-9d95-49ba-be21-d6edca0b7f31 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:18.873018 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_api:SELECT=97,nova_api:UPDATE=2,nova_api:INSERT=1 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:30:18.879063 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d8306486-9d95-49ba-be21-d6edca0b7f31 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:18.879551 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d8306486-9d95-49ba-be21-d6edca0b7f31 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:18.880020 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d8306486-9d95-49ba-be21-d6edca0b7f31 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:18.880724 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d8306486-9d95-49ba-be21-d6edca0b7f31 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:18.881180 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d8306486-9d95-49ba-be21-d6edca0b7f31 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:18.881620 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d8306486-9d95-49ba-be21-d6edca0b7f31 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:18.907437 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-914fd5ac-0a37-4d1c-9e8a-b12807fb4c5c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:18.908277 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-914fd5ac-0a37-4d1c-9e8a-b12807fb4c5c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: 48b3d4bf-6b20-4327-8f8d-521417405254] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:18.913607 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-d8306486-9d95-49ba-be21-d6edca0b7f31 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049458 Aug 08 02:30:18.919183 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 321/643] 10.222.0.27 () {58 vars in 1302 bytes} [Mon Aug 8 02:30:18 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:30:18.919355 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-914fd5ac-0a37-4d1c-9e8a-b12807fb4c5c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:18.919355 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-914fd5ac-0a37-4d1c-9e8a-b12807fb4c5c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:18.919355 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-914fd5ac-0a37-4d1c-9e8a-b12807fb4c5c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:18.919355 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-914fd5ac-0a37-4d1c-9e8a-b12807fb4c5c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:18.919729 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-914fd5ac-0a37-4d1c-9e8a-b12807fb4c5c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:18.919729 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-914fd5ac-0a37-4d1c-9e8a-b12807fb4c5c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:18.983253 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-914fd5ac-0a37-4d1c-9e8a-b12807fb4c5c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:18.990758 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-914fd5ac-0a37-4d1c-9e8a-b12807fb4c5c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254" status: 200 len: 1242 microversion: 2.1 time: 0.087258 Aug 08 02:30:18.991347 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 323/644] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:18 2022] GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254 => generated 1242 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:30:19.642922 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-c9c99ed6-c00d-4ecd-8e7d-4956ee374239 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:19.643649 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-c9c99ed6-c00d-4ecd-8e7d-4956ee374239 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:19.650333 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c9c99ed6-c00d-4ecd-8e7d-4956ee374239 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:19.651116 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c9c99ed6-c00d-4ecd-8e7d-4956ee374239 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:19.651669 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c9c99ed6-c00d-4ecd-8e7d-4956ee374239 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:19.652203 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c9c99ed6-c00d-4ecd-8e7d-4956ee374239 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:19.652658 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c9c99ed6-c00d-4ecd-8e7d-4956ee374239 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:19.653131 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c9c99ed6-c00d-4ecd-8e7d-4956ee374239 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:19.693980 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [req-3366a566-92e2-4b00-ae2f-5e35f89d5278 req-60cae7c0-9d9c-48e0-bab5-5b2e679901e0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "946fd571-bd98-44f9-89f4-52f0116b642d", "name": "network-vif-plugged", "status": "completed", "tag": "0bf885ea-ca94-45b2-aafa-7461404e6cf2"}]} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:30:19.706495 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-3366a566-92e2-4b00-ae2f-5e35f89d5278 req-60cae7c0-9d9c-48e0-bab5-5b2e679901e0 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:19.706980 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-3366a566-92e2-4b00-ae2f-5e35f89d5278 req-60cae7c0-9d9c-48e0-bab5-5b2e679901e0 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:19.707410 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-3366a566-92e2-4b00-ae2f-5e35f89d5278 req-60cae7c0-9d9c-48e0-bab5-5b2e679901e0 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:19.718834 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.compute.server_external_events [req-3366a566-92e2-4b00-ae2f-5e35f89d5278 req-60cae7c0-9d9c-48e0-bab5-5b2e679901e0 service nova] Creating event network-vif-plugged:0bf885ea-ca94-45b2-aafa-7461404e6cf2 for instance 946fd571-bd98-44f9-89f4-52f0116b642d on hv-cinder-74014 Aug 08 02:30:19.723900 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [req-3366a566-92e2-4b00-ae2f-5e35f89d5278 req-60cae7c0-9d9c-48e0-bab5-5b2e679901e0 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034034 Aug 08 02:30:19.724550 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 324/645] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:30:19 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 08 02:30:19.794805 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-c9c99ed6-c00d-4ecd-8e7d-4956ee374239 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:19.805780 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-c9c99ed6-c00d-4ecd-8e7d-4956ee374239 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:19.811264 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-c9c99ed6-c00d-4ecd-8e7d-4956ee374239 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:19.811264 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-c9c99ed6-c00d-4ecd-8e7d-4956ee374239 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1406 microversion: 2.20 time: 0.171294 Aug 08 02:30:19.811264 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 322/646] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:30:19 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1406 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:30:19.926346 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-de84bdf6-7dd5-4804-b0a0-41fae0d3f803 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:19.928849 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-de84bdf6-7dd5-4804-b0a0-41fae0d3f803 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:19.935835 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-de84bdf6-7dd5-4804-b0a0-41fae0d3f803 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:19.936303 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-de84bdf6-7dd5-4804-b0a0-41fae0d3f803 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:19.948339 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-de84bdf6-7dd5-4804-b0a0-41fae0d3f803 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:19.948339 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-de84bdf6-7dd5-4804-b0a0-41fae0d3f803 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:19.948339 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-de84bdf6-7dd5-4804-b0a0-41fae0d3f803 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:19.948339 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-de84bdf6-7dd5-4804-b0a0-41fae0d3f803 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:19.967094 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-de84bdf6-7dd5-4804-b0a0-41fae0d3f803 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045052 Aug 08 02:30:19.967648 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 325/647] 10.222.0.27 () {58 vars in 1302 bytes} [Mon Aug 8 02:30:19 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:30:20.011608 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-623c3190-28c1-4f8e-8fa8-87842a94366d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:20.012224 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-623c3190-28c1-4f8e-8fa8-87842a94366d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: 48b3d4bf-6b20-4327-8f8d-521417405254] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:20.019376 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-623c3190-28c1-4f8e-8fa8-87842a94366d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:20.019866 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-623c3190-28c1-4f8e-8fa8-87842a94366d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:20.020304 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-623c3190-28c1-4f8e-8fa8-87842a94366d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:20.021032 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-623c3190-28c1-4f8e-8fa8-87842a94366d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:20.021489 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-623c3190-28c1-4f8e-8fa8-87842a94366d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:20.021914 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-623c3190-28c1-4f8e-8fa8-87842a94366d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:20.086636 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-623c3190-28c1-4f8e-8fa8-87842a94366d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:20.094844 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-623c3190-28c1-4f8e-8fa8-87842a94366d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254" status: 200 len: 1290 microversion: 2.1 time: 0.087293 Aug 08 02:30:20.095407 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 323/648] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:20 2022] GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254 => generated 1290 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:20.832326 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-21e2a9a3-5423-4408-9879-edbe97e889ea tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:20.832976 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-21e2a9a3-5423-4408-9879-edbe97e889ea tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:20.840561 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-21e2a9a3-5423-4408-9879-edbe97e889ea tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:20.841292 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-21e2a9a3-5423-4408-9879-edbe97e889ea tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:20.844557 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-21e2a9a3-5423-4408-9879-edbe97e889ea tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:20.844557 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-21e2a9a3-5423-4408-9879-edbe97e889ea tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:20.844557 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-21e2a9a3-5423-4408-9879-edbe97e889ea tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:20.844557 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-21e2a9a3-5423-4408-9879-edbe97e889ea tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:20.980890 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-d53b68e7-8eba-4042-9cfc-390766ac09af tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:20.983172 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-d53b68e7-8eba-4042-9cfc-390766ac09af tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:20.994832 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d53b68e7-8eba-4042-9cfc-390766ac09af tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:20.997295 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d53b68e7-8eba-4042-9cfc-390766ac09af tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:20.998032 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d53b68e7-8eba-4042-9cfc-390766ac09af tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:20.999016 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d53b68e7-8eba-4042-9cfc-390766ac09af tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:20.999666 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d53b68e7-8eba-4042-9cfc-390766ac09af tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:21.000519 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d53b68e7-8eba-4042-9cfc-390766ac09af tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:21.036161 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-d53b68e7-8eba-4042-9cfc-390766ac09af tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.060556 Aug 08 02:30:21.037257 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 324/649] 10.222.0.27 () {58 vars in 1302 bytes} [Mon Aug 8 02:30:20 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 08 02:30:21.050339 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-21e2a9a3-5423-4408-9879-edbe97e889ea tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:21.059825 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-21e2a9a3-5423-4408-9879-edbe97e889ea tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:21.061910 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-21e2a9a3-5423-4408-9879-edbe97e889ea tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:21.063209 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-21e2a9a3-5423-4408-9879-edbe97e889ea tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1406 microversion: 2.20 time: 0.234973 Aug 08 02:30:21.063770 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 326/650] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:30:20 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1406 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:30:21.127765 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-95ae4e26-8a76-4dfc-b660-56ace3ccdea2 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:21.128386 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-95ae4e26-8a76-4dfc-b660-56ace3ccdea2 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: 48b3d4bf-6b20-4327-8f8d-521417405254] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:21.134454 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-95ae4e26-8a76-4dfc-b660-56ace3ccdea2 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:21.134939 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-95ae4e26-8a76-4dfc-b660-56ace3ccdea2 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:21.136045 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-95ae4e26-8a76-4dfc-b660-56ace3ccdea2 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:21.136793 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-95ae4e26-8a76-4dfc-b660-56ace3ccdea2 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:21.137270 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-95ae4e26-8a76-4dfc-b660-56ace3ccdea2 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:21.137698 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-95ae4e26-8a76-4dfc-b660-56ace3ccdea2 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:21.199372 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-95ae4e26-8a76-4dfc-b660-56ace3ccdea2 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:21.209799 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-95ae4e26-8a76-4dfc-b660-56ace3ccdea2 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254" status: 200 len: 1296 microversion: 2.1 time: 0.085937 Aug 08 02:30:21.210370 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 325/651] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:21 2022] GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254 => generated 1296 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:22.049030 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-abfd5962-54c3-42b2-85ee-99832a18aa74 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:22.051137 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-abfd5962-54c3-42b2-85ee-99832a18aa74 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:22.058561 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-abfd5962-54c3-42b2-85ee-99832a18aa74 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:22.059037 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-abfd5962-54c3-42b2-85ee-99832a18aa74 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:22.059475 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-abfd5962-54c3-42b2-85ee-99832a18aa74 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:22.060186 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-abfd5962-54c3-42b2-85ee-99832a18aa74 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:22.060637 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-abfd5962-54c3-42b2-85ee-99832a18aa74 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:22.061752 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-abfd5962-54c3-42b2-85ee-99832a18aa74 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:22.086408 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-032b0aaa-581b-4f8e-b346-68f744730f1a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:22.087021 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-032b0aaa-581b-4f8e-b346-68f744730f1a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:22.095328 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-032b0aaa-581b-4f8e-b346-68f744730f1a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:22.095806 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-032b0aaa-581b-4f8e-b346-68f744730f1a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:22.096269 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-032b0aaa-581b-4f8e-b346-68f744730f1a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:22.097211 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-032b0aaa-581b-4f8e-b346-68f744730f1a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:22.097667 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-032b0aaa-581b-4f8e-b346-68f744730f1a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:22.098105 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-032b0aaa-581b-4f8e-b346-68f744730f1a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:22.110856 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-abfd5962-54c3-42b2-85ee-99832a18aa74 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.065849 Aug 08 02:30:22.112011 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 327/652] 10.222.0.27 () {58 vars in 1302 bytes} [Mon Aug 8 02:30:22 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24/os-volume_attachments => generated 197 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:30:22.234382 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-49eeb38e-c262-42b5-96bf-8c2eac8647c5 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:22.235301 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-49eeb38e-c262-42b5-96bf-8c2eac8647c5 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: 48b3d4bf-6b20-4327-8f8d-521417405254] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:22.243518 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-032b0aaa-581b-4f8e-b346-68f744730f1a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:22.243985 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-49eeb38e-c262-42b5-96bf-8c2eac8647c5 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:22.245362 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-49eeb38e-c262-42b5-96bf-8c2eac8647c5 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:22.246148 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-49eeb38e-c262-42b5-96bf-8c2eac8647c5 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:22.247706 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-49eeb38e-c262-42b5-96bf-8c2eac8647c5 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:22.248956 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-49eeb38e-c262-42b5-96bf-8c2eac8647c5 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:22.250008 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-49eeb38e-c262-42b5-96bf-8c2eac8647c5 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:22.255123 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-032b0aaa-581b-4f8e-b346-68f744730f1a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:22.257008 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-032b0aaa-581b-4f8e-b346-68f744730f1a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:22.258443 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-032b0aaa-581b-4f8e-b346-68f744730f1a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1406 microversion: 2.20 time: 0.175671 Aug 08 02:30:22.259358 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 326/653] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:30:22 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1406 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:30:22.397793 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-49eeb38e-c262-42b5-96bf-8c2eac8647c5 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:22.406687 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-49eeb38e-c262-42b5-96bf-8c2eac8647c5 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254" status: 200 len: 1364 microversion: 2.1 time: 0.178639 Aug 08 02:30:22.407354 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 328/654] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:22 2022] GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254 => generated 1364 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:30:22.850429 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [req-a1d9721a-f89a-4bdd-8629-00cb218ecff4 req-ca822806-9493-4015-883a-61186823fd39 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "48b3d4bf-6b20-4327-8f8d-521417405254", "tag": "961375ea-c683-4138-905d-e3bf380590df"}]} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:30:22.861009 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-a1d9721a-f89a-4bdd-8629-00cb218ecff4 req-ca822806-9493-4015-883a-61186823fd39 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:22.861499 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-a1d9721a-f89a-4bdd-8629-00cb218ecff4 req-ca822806-9493-4015-883a-61186823fd39 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:22.861958 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-a1d9721a-f89a-4bdd-8629-00cb218ecff4 req-ca822806-9493-4015-883a-61186823fd39 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:22.875719 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.compute.server_external_events [req-a1d9721a-f89a-4bdd-8629-00cb218ecff4 req-ca822806-9493-4015-883a-61186823fd39 service nova] Creating event network-changed:961375ea-c683-4138-905d-e3bf380590df for instance 48b3d4bf-6b20-4327-8f8d-521417405254 on hv-cinder-74014 Aug 08 02:30:22.881181 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [req-a1d9721a-f89a-4bdd-8629-00cb218ecff4 req-ca822806-9493-4015-883a-61186823fd39 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034293 Aug 08 02:30:22.881994 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 327/655] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:30:22 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:30:23.124170 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-f0c8e0fd-1045-48a6-9897-73d71bb057e1 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:23.126243 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-f0c8e0fd-1045-48a6-9897-73d71bb057e1 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:23.132774 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f0c8e0fd-1045-48a6-9897-73d71bb057e1 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:23.133310 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f0c8e0fd-1045-48a6-9897-73d71bb057e1 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:23.133747 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f0c8e0fd-1045-48a6-9897-73d71bb057e1 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:23.134442 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f0c8e0fd-1045-48a6-9897-73d71bb057e1 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:23.134873 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f0c8e0fd-1045-48a6-9897-73d71bb057e1 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:23.136109 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f0c8e0fd-1045-48a6-9897-73d71bb057e1 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:23.164016 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-f0c8e0fd-1045-48a6-9897-73d71bb057e1 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044564 Aug 08 02:30:23.164559 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 329/656] 10.222.0.27 () {58 vars in 1302 bytes} [Mon Aug 8 02:30:23 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:30:23.282406 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-43c93cf9-dd2c-4ab7-a1c3-7ab9aa40f343 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:23.283135 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-43c93cf9-dd2c-4ab7-a1c3-7ab9aa40f343 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:23.289307 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-43c93cf9-dd2c-4ab7-a1c3-7ab9aa40f343 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:23.289788 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-43c93cf9-dd2c-4ab7-a1c3-7ab9aa40f343 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:23.290223 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-43c93cf9-dd2c-4ab7-a1c3-7ab9aa40f343 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:23.290904 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-43c93cf9-dd2c-4ab7-a1c3-7ab9aa40f343 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:23.291619 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-43c93cf9-dd2c-4ab7-a1c3-7ab9aa40f343 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:23.292074 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-43c93cf9-dd2c-4ab7-a1c3-7ab9aa40f343 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:23.424248 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-43c93cf9-dd2c-4ab7-a1c3-7ab9aa40f343 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:23.427614 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-3214ce5b-2ebe-4b8f-8f0b-a86ebd98cbf4 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:23.428301 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-3214ce5b-2ebe-4b8f-8f0b-a86ebd98cbf4 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: 48b3d4bf-6b20-4327-8f8d-521417405254] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:23.435715 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3214ce5b-2ebe-4b8f-8f0b-a86ebd98cbf4 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:23.436232 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3214ce5b-2ebe-4b8f-8f0b-a86ebd98cbf4 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:23.436695 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3214ce5b-2ebe-4b8f-8f0b-a86ebd98cbf4 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:23.437451 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3214ce5b-2ebe-4b8f-8f0b-a86ebd98cbf4 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:23.437914 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3214ce5b-2ebe-4b8f-8f0b-a86ebd98cbf4 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:23.438042 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-43c93cf9-dd2c-4ab7-a1c3-7ab9aa40f343 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:23.438351 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3214ce5b-2ebe-4b8f-8f0b-a86ebd98cbf4 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:23.439775 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-43c93cf9-dd2c-4ab7-a1c3-7ab9aa40f343 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:23.441472 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-43c93cf9-dd2c-4ab7-a1c3-7ab9aa40f343 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1599 microversion: 2.20 time: 0.163892 Aug 08 02:30:23.442281 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 328/657] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:30:23 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1599 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:30:23.466049 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-76295e9c-9cc3-49bb-bbc4-9d96132d821f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:23.466693 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-76295e9c-9cc3-49bb-bbc4-9d96132d821f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:23.473586 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-76295e9c-9cc3-49bb-bbc4-9d96132d821f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:23.474658 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-76295e9c-9cc3-49bb-bbc4-9d96132d821f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:23.475248 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-76295e9c-9cc3-49bb-bbc4-9d96132d821f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:23.476068 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-76295e9c-9cc3-49bb-bbc4-9d96132d821f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:23.476607 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-76295e9c-9cc3-49bb-bbc4-9d96132d821f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:23.477311 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-76295e9c-9cc3-49bb-bbc4-9d96132d821f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:23.574294 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-76295e9c-9cc3-49bb-bbc4-9d96132d821f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d/os-interface" status: 200 len: 285 microversion: 2.20 time: 0.112652 Aug 08 02:30:23.574902 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 329/658] 10.222.0.27 () {60 vars in 1316 bytes} [Mon Aug 8 02:30:23 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d/os-interface => generated 285 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 08 02:30:23.616115 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-3214ce5b-2ebe-4b8f-8f0b-a86ebd98cbf4 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:23.625609 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-3214ce5b-2ebe-4b8f-8f0b-a86ebd98cbf4 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254" status: 200 len: 1364 microversion: 2.1 time: 0.202106 Aug 08 02:30:23.626191 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 330/659] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:23 2022] GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254 => generated 1364 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:30:24.175695 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-ffcc191a-cf25-40fb-bbee-85d5994e948a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:24.178032 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-ffcc191a-cf25-40fb-bbee-85d5994e948a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:24.184616 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ffcc191a-cf25-40fb-bbee-85d5994e948a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:24.185123 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ffcc191a-cf25-40fb-bbee-85d5994e948a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:24.185557 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ffcc191a-cf25-40fb-bbee-85d5994e948a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:24.186299 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ffcc191a-cf25-40fb-bbee-85d5994e948a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:24.186748 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ffcc191a-cf25-40fb-bbee-85d5994e948a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:24.187166 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ffcc191a-cf25-40fb-bbee-85d5994e948a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:24.218825 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-ffcc191a-cf25-40fb-bbee-85d5994e948a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047104 Aug 08 02:30:24.219400 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 330/660] 10.222.0.27 () {58 vars in 1302 bytes} [Mon Aug 8 02:30:24 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:30:24.646473 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-262210f6-8e51-44be-9b4f-e565c89fcf11 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:24.647128 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-262210f6-8e51-44be-9b4f-e565c89fcf11 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: 48b3d4bf-6b20-4327-8f8d-521417405254] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:24.653688 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-262210f6-8e51-44be-9b4f-e565c89fcf11 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:24.654182 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-262210f6-8e51-44be-9b4f-e565c89fcf11 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:24.654627 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-262210f6-8e51-44be-9b4f-e565c89fcf11 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:24.655325 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-262210f6-8e51-44be-9b4f-e565c89fcf11 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:24.655794 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-262210f6-8e51-44be-9b4f-e565c89fcf11 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:24.656226 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-262210f6-8e51-44be-9b4f-e565c89fcf11 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:24.794483 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-262210f6-8e51-44be-9b4f-e565c89fcf11 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:24.807564 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-262210f6-8e51-44be-9b4f-e565c89fcf11 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254" status: 200 len: 1364 microversion: 2.1 time: 0.164459 Aug 08 02:30:24.808172 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 331/661] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:24 2022] GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254 => generated 1364 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:24.897288 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [req-a283430a-c30d-41bb-a061-7fa5b6f4e346 req-00636dd4-4233-4334-9c47-495ce867bc1c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "946fd571-bd98-44f9-89f4-52f0116b642d", "tag": "0bf885ea-ca94-45b2-aafa-7461404e6cf2"}]} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:30:24.907809 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-a283430a-c30d-41bb-a061-7fa5b6f4e346 req-00636dd4-4233-4334-9c47-495ce867bc1c service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:24.908529 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-a283430a-c30d-41bb-a061-7fa5b6f4e346 req-00636dd4-4233-4334-9c47-495ce867bc1c service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:24.908997 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-a283430a-c30d-41bb-a061-7fa5b6f4e346 req-00636dd4-4233-4334-9c47-495ce867bc1c service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:24.921380 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.compute.server_external_events [req-a283430a-c30d-41bb-a061-7fa5b6f4e346 req-00636dd4-4233-4334-9c47-495ce867bc1c service nova] Creating event network-changed:0bf885ea-ca94-45b2-aafa-7461404e6cf2 for instance 946fd571-bd98-44f9-89f4-52f0116b642d on hv-cinder-74014 Aug 08 02:30:24.926405 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [req-a283430a-c30d-41bb-a061-7fa5b6f4e346 req-00636dd4-4233-4334-9c47-495ce867bc1c service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032824 Aug 08 02:30:24.926989 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 331/662] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:30:24 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:30:25.232310 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-00d455fb-128c-4172-bf0d-0ff4925725a8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:25.234481 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-00d455fb-128c-4172-bf0d-0ff4925725a8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:25.240406 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-00d455fb-128c-4172-bf0d-0ff4925725a8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:25.240929 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-00d455fb-128c-4172-bf0d-0ff4925725a8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:25.241372 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-00d455fb-128c-4172-bf0d-0ff4925725a8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:25.242067 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-00d455fb-128c-4172-bf0d-0ff4925725a8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:25.242525 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-00d455fb-128c-4172-bf0d-0ff4925725a8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:25.242952 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-00d455fb-128c-4172-bf0d-0ff4925725a8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:25.272024 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-00d455fb-128c-4172-bf0d-0ff4925725a8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044368 Aug 08 02:30:25.272595 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 332/663] 10.222.0.27 () {58 vars in 1302 bytes} [Mon Aug 8 02:30:25 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:30:25.828847 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-f9e049db-f9b6-416d-9be8-86977d874e2d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:25.834691 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-f9e049db-f9b6-416d-9be8-86977d874e2d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: 48b3d4bf-6b20-4327-8f8d-521417405254] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:25.835783 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f9e049db-f9b6-416d-9be8-86977d874e2d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:25.836275 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f9e049db-f9b6-416d-9be8-86977d874e2d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:25.836737 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f9e049db-f9b6-416d-9be8-86977d874e2d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:25.837438 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f9e049db-f9b6-416d-9be8-86977d874e2d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:25.837923 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f9e049db-f9b6-416d-9be8-86977d874e2d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:25.839122 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f9e049db-f9b6-416d-9be8-86977d874e2d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:25.983317 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-f9e049db-f9b6-416d-9be8-86977d874e2d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:25.992493 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-f9e049db-f9b6-416d-9be8-86977d874e2d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254" status: 200 len: 1364 microversion: 2.1 time: 0.168082 Aug 08 02:30:25.996706 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 332/664] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:25 2022] GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254 => generated 1364 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:26.284473 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-1aebac78-5624-4408-993c-91ea32ba1625 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:26.286699 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-1aebac78-5624-4408-993c-91ea32ba1625 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:26.293772 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1aebac78-5624-4408-993c-91ea32ba1625 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:26.294260 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1aebac78-5624-4408-993c-91ea32ba1625 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:26.294699 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1aebac78-5624-4408-993c-91ea32ba1625 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:26.295392 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1aebac78-5624-4408-993c-91ea32ba1625 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:26.295875 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1aebac78-5624-4408-993c-91ea32ba1625 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:26.296299 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1aebac78-5624-4408-993c-91ea32ba1625 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:26.329016 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-1aebac78-5624-4408-993c-91ea32ba1625 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.048597 Aug 08 02:30:26.329579 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 333/665] 10.222.0.27 () {58 vars in 1302 bytes} [Mon Aug 8 02:30:26 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24/os-volume_attachments => generated 25 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Aug 08 02:30:26.408749 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-2701a995-a278-4f30-b02e-4667e8086a53 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:26.409531 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-2701a995-a278-4f30-b02e-4667e8086a53 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:26.416757 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2701a995-a278-4f30-b02e-4667e8086a53 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:26.418301 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2701a995-a278-4f30-b02e-4667e8086a53 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:26.419165 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2701a995-a278-4f30-b02e-4667e8086a53 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:26.424585 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2701a995-a278-4f30-b02e-4667e8086a53 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:26.424585 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2701a995-a278-4f30-b02e-4667e8086a53 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:26.424585 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2701a995-a278-4f30-b02e-4667e8086a53 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:26.450175 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-2701a995-a278-4f30-b02e-4667e8086a53 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Going to try to terminate instance {{(pid=118351) delete /opt/stack/nova/nova/compute/api.py:2666}} Aug 08 02:30:26.802982 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-2701a995-a278-4f30-b02e-4667e8086a53 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "DELETE /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 204 len: 0 microversion: 2.1 time: 0.398015 Aug 08 02:30:26.803865 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 333/666] 10.222.0.27 () {58 vars in 1239 bytes} [Mon Aug 8 02:30:26 2022] DELETE /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 08 02:30:26.812437 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-e2472394-155c-4189-8e69-25f1e2347114 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:26.813081 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-e2472394-155c-4189-8e69-25f1e2347114 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:26.819867 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e2472394-155c-4189-8e69-25f1e2347114 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:26.820494 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e2472394-155c-4189-8e69-25f1e2347114 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:26.821165 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e2472394-155c-4189-8e69-25f1e2347114 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:26.821977 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e2472394-155c-4189-8e69-25f1e2347114 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:26.822730 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e2472394-155c-4189-8e69-25f1e2347114 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:26.823414 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e2472394-155c-4189-8e69-25f1e2347114 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:26.844612 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell1:SELECT=100 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:30:26.966903 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-e2472394-155c-4189-8e69-25f1e2347114 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:26.974553 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-e2472394-155c-4189-8e69-25f1e2347114 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1690 microversion: 2.1 time: 0.166099 Aug 08 02:30:26.975111 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 334/667] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:26 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1690 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:27.015288 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-e6171d94-86ee-4103-921f-3f3f37c3426f tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:27.016603 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-e6171d94-86ee-4103-921f-3f3f37c3426f tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: 48b3d4bf-6b20-4327-8f8d-521417405254] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:27.023365 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e6171d94-86ee-4103-921f-3f3f37c3426f tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:27.024043 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e6171d94-86ee-4103-921f-3f3f37c3426f tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:27.024485 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e6171d94-86ee-4103-921f-3f3f37c3426f tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:27.025392 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e6171d94-86ee-4103-921f-3f3f37c3426f tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:27.026369 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e6171d94-86ee-4103-921f-3f3f37c3426f tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:27.026802 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e6171d94-86ee-4103-921f-3f3f37c3426f tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:27.162181 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-e6171d94-86ee-4103-921f-3f3f37c3426f tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:27.170035 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-e6171d94-86ee-4103-921f-3f3f37c3426f tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254" status: 200 len: 1364 microversion: 2.1 time: 0.159627 Aug 08 02:30:27.170599 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 334/668] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:27 2022] GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254 => generated 1364 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:27.997349 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell0:SELECT=6 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:30:27.998474 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell0:SELECT=24 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:30:28.054116 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-1b499db3-3537-4860-9849-c3feff8960af tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:28.054748 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-1b499db3-3537-4860-9849-c3feff8960af tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:28.061653 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1b499db3-3537-4860-9849-c3feff8960af tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:28.062149 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1b499db3-3537-4860-9849-c3feff8960af tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:28.062614 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1b499db3-3537-4860-9849-c3feff8960af tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:28.063381 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1b499db3-3537-4860-9849-c3feff8960af tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:28.063837 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1b499db3-3537-4860-9849-c3feff8960af tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:28.064270 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1b499db3-3537-4860-9849-c3feff8960af tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:28.190574 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-80fc7049-cb12-43dd-abf9-56eaea5c061b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:28.191184 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-80fc7049-cb12-43dd-abf9-56eaea5c061b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: 48b3d4bf-6b20-4327-8f8d-521417405254] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:28.193361 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell0:SELECT=6 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:30:28.197538 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-1b499db3-3537-4860-9849-c3feff8960af tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:28.199017 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-80fc7049-cb12-43dd-abf9-56eaea5c061b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:28.199506 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-80fc7049-cb12-43dd-abf9-56eaea5c061b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:28.199935 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-80fc7049-cb12-43dd-abf9-56eaea5c061b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:28.200626 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-80fc7049-cb12-43dd-abf9-56eaea5c061b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:28.201103 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-80fc7049-cb12-43dd-abf9-56eaea5c061b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:28.201527 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-80fc7049-cb12-43dd-abf9-56eaea5c061b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:28.205546 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-1b499db3-3537-4860-9849-c3feff8960af tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1690 microversion: 2.1 time: 0.214075 Aug 08 02:30:28.206125 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 335/669] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:27 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1690 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:28.336722 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-80fc7049-cb12-43dd-abf9-56eaea5c061b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:28.345232 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-80fc7049-cb12-43dd-abf9-56eaea5c061b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254" status: 200 len: 1364 microversion: 2.1 time: 0.158429 Aug 08 02:30:28.345804 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 335/670] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:28 2022] GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254 => generated 1364 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:30:28.375901 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-911a0183-7ed6-4571-b56f-38eade82eff5 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:28.376407 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-911a0183-7ed6-4571-b56f-38eade82eff5 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: aa20d25c-bd00-41e6-b350-309368839578] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:28.382471 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-911a0183-7ed6-4571-b56f-38eade82eff5 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:28.382961 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-911a0183-7ed6-4571-b56f-38eade82eff5 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:28.383403 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-911a0183-7ed6-4571-b56f-38eade82eff5 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:28.384137 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-911a0183-7ed6-4571-b56f-38eade82eff5 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:28.384599 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-911a0183-7ed6-4571-b56f-38eade82eff5 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:28.385086 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-911a0183-7ed6-4571-b56f-38eade82eff5 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:28.408481 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-911a0183-7ed6-4571-b56f-38eade82eff5 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578/ips" status: 200 len: 148 microversion: 2.1 time: 0.088260 Aug 08 02:30:28.409142 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 336/671] 10.222.0.27 () {58 vars in 1248 bytes} [Mon Aug 8 02:30:28 2022] GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578/ips => generated 148 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 08 02:30:29.227497 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-73730bd7-5df9-4c16-a06c-72e61ca351eb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:29.228118 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-73730bd7-5df9-4c16-a06c-72e61ca351eb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:29.236474 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-73730bd7-5df9-4c16-a06c-72e61ca351eb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:29.237000 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-73730bd7-5df9-4c16-a06c-72e61ca351eb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:29.237437 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-73730bd7-5df9-4c16-a06c-72e61ca351eb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:29.238134 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-73730bd7-5df9-4c16-a06c-72e61ca351eb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:29.238587 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-73730bd7-5df9-4c16-a06c-72e61ca351eb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:29.239014 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-73730bd7-5df9-4c16-a06c-72e61ca351eb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:29.366581 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-982459a5-8d58-48d9-905a-fcd796273e4e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:29.367200 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-982459a5-8d58-48d9-905a-fcd796273e4e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: 48b3d4bf-6b20-4327-8f8d-521417405254] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:29.374344 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-982459a5-8d58-48d9-905a-fcd796273e4e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:29.374835 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-982459a5-8d58-48d9-905a-fcd796273e4e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:29.375280 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-982459a5-8d58-48d9-905a-fcd796273e4e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:29.376011 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-982459a5-8d58-48d9-905a-fcd796273e4e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:29.376471 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-982459a5-8d58-48d9-905a-fcd796273e4e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:29.376954 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-982459a5-8d58-48d9-905a-fcd796273e4e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:29.381616 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-73730bd7-5df9-4c16-a06c-72e61ca351eb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:29.385666 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:30:29.391258 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-73730bd7-5df9-4c16-a06c-72e61ca351eb tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1690 microversion: 2.1 time: 0.167964 Aug 08 02:30:29.391868 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 336/672] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:29 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1690 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:29.514609 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-982459a5-8d58-48d9-905a-fcd796273e4e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:29.522102 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-982459a5-8d58-48d9-905a-fcd796273e4e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254" status: 200 len: 1364 microversion: 2.1 time: 0.159611 Aug 08 02:30:29.522667 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 337/673] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:29 2022] GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254 => generated 1364 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:30:30.413141 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-6baeccde-49d3-4fe1-b08c-4555ff5e5596 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:30.414086 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-6baeccde-49d3-4fe1-b08c-4555ff5e5596 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:30.422627 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6baeccde-49d3-4fe1-b08c-4555ff5e5596 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:30.427375 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6baeccde-49d3-4fe1-b08c-4555ff5e5596 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:30.427375 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6baeccde-49d3-4fe1-b08c-4555ff5e5596 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:30.427375 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6baeccde-49d3-4fe1-b08c-4555ff5e5596 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:30.427375 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6baeccde-49d3-4fe1-b08c-4555ff5e5596 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:30.427772 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6baeccde-49d3-4fe1-b08c-4555ff5e5596 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:30.562093 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-6baeccde-49d3-4fe1-b08c-4555ff5e5596 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:30.570771 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-6baeccde-49d3-4fe1-b08c-4555ff5e5596 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1690 microversion: 2.1 time: 0.162808 Aug 08 02:30:30.571750 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 337/674] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:30 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1690 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:30.603939 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-a715d6f9-eb5a-4b7e-a56e-837b3546c6fb tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:30.604559 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-a715d6f9-eb5a-4b7e-a56e-837b3546c6fb tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: 48b3d4bf-6b20-4327-8f8d-521417405254] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:30.611031 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a715d6f9-eb5a-4b7e-a56e-837b3546c6fb tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:30.611528 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a715d6f9-eb5a-4b7e-a56e-837b3546c6fb tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:30.611974 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a715d6f9-eb5a-4b7e-a56e-837b3546c6fb tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:30.612693 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a715d6f9-eb5a-4b7e-a56e-837b3546c6fb tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:30.613157 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a715d6f9-eb5a-4b7e-a56e-837b3546c6fb tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:30.613589 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a715d6f9-eb5a-4b7e-a56e-837b3546c6fb tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:30.747769 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-a715d6f9-eb5a-4b7e-a56e-837b3546c6fb tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:30.755293 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-a715d6f9-eb5a-4b7e-a56e-837b3546c6fb tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254" status: 200 len: 1364 microversion: 2.1 time: 0.215822 Aug 08 02:30:30.755860 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 338/675] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:30 2022] GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254 => generated 1364 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:30:31.149857 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-d21f134f-66d9-4d8a-a896-44f66862dfc8 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "09091e4c-3641-47bb-bdb7-14d386fcdcad"}} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:30:31.152364 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-d21f134f-66d9-4d8a-a896-44f66862dfc8 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: aa20d25c-bd00-41e6-b350-309368839578] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:31.159011 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d21f134f-66d9-4d8a-a896-44f66862dfc8 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:31.159522 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d21f134f-66d9-4d8a-a896-44f66862dfc8 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:31.159966 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d21f134f-66d9-4d8a-a896-44f66862dfc8 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:31.160693 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d21f134f-66d9-4d8a-a896-44f66862dfc8 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:31.161151 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d21f134f-66d9-4d8a-a896-44f66862dfc8 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:31.161585 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d21f134f-66d9-4d8a-a896-44f66862dfc8 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:31.595437 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-7327231a-98f3-4147-ba1b-3d3b75fd564c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:31.599548 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-7327231a-98f3-4147-ba1b-3d3b75fd564c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:31.605140 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7327231a-98f3-4147-ba1b-3d3b75fd564c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:31.605778 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7327231a-98f3-4147-ba1b-3d3b75fd564c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:31.606062 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7327231a-98f3-4147-ba1b-3d3b75fd564c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:31.606771 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7327231a-98f3-4147-ba1b-3d3b75fd564c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:31.607230 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7327231a-98f3-4147-ba1b-3d3b75fd564c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:31.607663 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7327231a-98f3-4147-ba1b-3d3b75fd564c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:31.750399 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-7327231a-98f3-4147-ba1b-3d3b75fd564c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:31.758344 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-7327231a-98f3-4147-ba1b-3d3b75fd564c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1690 microversion: 2.1 time: 0.166807 Aug 08 02:30:31.758916 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 339/676] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:31 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1690 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:31.777147 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-f3c0f6bf-ae3d-4366-aaca-30a3c8de516e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:31.777745 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-f3c0f6bf-ae3d-4366-aaca-30a3c8de516e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: 48b3d4bf-6b20-4327-8f8d-521417405254] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:31.783781 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f3c0f6bf-ae3d-4366-aaca-30a3c8de516e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:31.784259 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f3c0f6bf-ae3d-4366-aaca-30a3c8de516e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:31.784715 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f3c0f6bf-ae3d-4366-aaca-30a3c8de516e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:31.785405 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f3c0f6bf-ae3d-4366-aaca-30a3c8de516e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:31.785852 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f3c0f6bf-ae3d-4366-aaca-30a3c8de516e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:31.786274 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f3c0f6bf-ae3d-4366-aaca-30a3c8de516e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:31.931130 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-f3c0f6bf-ae3d-4366-aaca-30a3c8de516e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:31.941844 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-f3c0f6bf-ae3d-4366-aaca-30a3c8de516e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254" status: 200 len: 1364 microversion: 2.1 time: 0.169048 Aug 08 02:30:31.942411 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 340/677] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:31 2022] GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254 => generated 1364 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:30:32.781190 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-4f8e7ab5-da08-4135-ad05-274c2b47b6f2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:32.781861 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-4f8e7ab5-da08-4135-ad05-274c2b47b6f2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:32.788167 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4f8e7ab5-da08-4135-ad05-274c2b47b6f2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:32.788658 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4f8e7ab5-da08-4135-ad05-274c2b47b6f2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:32.789321 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4f8e7ab5-da08-4135-ad05-274c2b47b6f2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:32.790003 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4f8e7ab5-da08-4135-ad05-274c2b47b6f2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:32.790458 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4f8e7ab5-da08-4135-ad05-274c2b47b6f2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:32.790888 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4f8e7ab5-da08-4135-ad05-274c2b47b6f2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:32.859115 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-4f8e7ab5-da08-4135-ad05-274c2b47b6f2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:32.867840 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-4f8e7ab5-da08-4135-ad05-274c2b47b6f2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1622 microversion: 2.1 time: 0.090590 Aug 08 02:30:32.868420 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 341/678] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:32 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1622 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:32.931310 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-d21f134f-66d9-4d8a-a896-44f66862dfc8 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "POST /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.770743 Aug 08 02:30:32.931310 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 338/679] 10.222.0.27 () {60 vars in 1323 bytes} [Mon Aug 8 02:30:31 2022] POST /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578/os-volume_attachments => generated 194 bytes in 1772 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:30:32.966044 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-4f860411-f379-4461-b5a5-f479a1455a61 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:32.966044 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-4f860411-f379-4461-b5a5-f479a1455a61 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: 48b3d4bf-6b20-4327-8f8d-521417405254] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:32.972968 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4f860411-f379-4461-b5a5-f479a1455a61 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:32.973642 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4f860411-f379-4461-b5a5-f479a1455a61 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:32.975784 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4f860411-f379-4461-b5a5-f479a1455a61 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:32.977257 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4f860411-f379-4461-b5a5-f479a1455a61 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:32.978499 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4f860411-f379-4461-b5a5-f479a1455a61 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:32.979321 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4f860411-f379-4461-b5a5-f479a1455a61 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:33.000874 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [req-1620555a-3182-4da8-9afb-200985e10a60 req-7d151ed9-c0a7-42fa-b881-a386d019ca70 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a5b1155c-5bba-46ae-8f3c-61f70c8a8a24", "name": "network-vif-deleted", "tag": "9b3bb00c-894f-4b64-935c-89a64519fd90"}]} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:30:33.014693 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-1620555a-3182-4da8-9afb-200985e10a60 req-7d151ed9-c0a7-42fa-b881-a386d019ca70 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:33.015415 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-1620555a-3182-4da8-9afb-200985e10a60 req-7d151ed9-c0a7-42fa-b881-a386d019ca70 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:33.016033 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-1620555a-3182-4da8-9afb-200985e10a60 req-7d151ed9-c0a7-42fa-b881-a386d019ca70 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:33.033193 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.compute.server_external_events [req-1620555a-3182-4da8-9afb-200985e10a60 req-7d151ed9-c0a7-42fa-b881-a386d019ca70 service nova] Creating event network-vif-deleted:9b3bb00c-894f-4b64-935c-89a64519fd90 for instance a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 on hv-cinder-74014 Aug 08 02:30:33.039502 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [req-1620555a-3182-4da8-9afb-200985e10a60 req-7d151ed9-c0a7-42fa-b881-a386d019ca70 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043142 Aug 08 02:30:33.041447 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 339/680] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:30:32 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 08 02:30:33.239420 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-4f860411-f379-4461-b5a5-f479a1455a61 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:33.253490 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-4f860411-f379-4461-b5a5-f479a1455a61 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254" status: 200 len: 1364 microversion: 2.1 time: 0.293041 Aug 08 02:30:33.254289 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 342/681] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:32 2022] GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254 => generated 1364 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:33.891574 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-0acf04b4-d19a-4fbb-b904-1ea7385507b8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:33.892282 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-0acf04b4-d19a-4fbb-b904-1ea7385507b8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:33.900659 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0acf04b4-d19a-4fbb-b904-1ea7385507b8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:33.903005 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0acf04b4-d19a-4fbb-b904-1ea7385507b8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:33.903551 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0acf04b4-d19a-4fbb-b904-1ea7385507b8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:33.904319 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0acf04b4-d19a-4fbb-b904-1ea7385507b8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:33.904870 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0acf04b4-d19a-4fbb-b904-1ea7385507b8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:33.905309 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0acf04b4-d19a-4fbb-b904-1ea7385507b8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:33.951269 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0acf04b4-d19a-4fbb-b904-1ea7385507b8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:33.951905 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0acf04b4-d19a-4fbb-b904-1ea7385507b8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:33.952174 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0acf04b4-d19a-4fbb-b904-1ea7385507b8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:33.961014 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.objects.instance [None req-0acf04b4-d19a-4fbb-b904-1ea7385507b8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lazy-loading 'fault' on Instance uuid a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 {{(pid=118351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 08 02:30:34.040165 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-0acf04b4-d19a-4fbb-b904-1ea7385507b8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:34.051722 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-0acf04b4-d19a-4fbb-b904-1ea7385507b8 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 200 len: 1336 microversion: 2.1 time: 0.164930 Aug 08 02:30:34.060751 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 340/682] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:33 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 1336 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:34.275397 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-8c689e68-df97-4e4f-a445-ec44e9e79719 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:34.277323 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-8c689e68-df97-4e4f-a445-ec44e9e79719 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: 48b3d4bf-6b20-4327-8f8d-521417405254] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:34.285112 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8c689e68-df97-4e4f-a445-ec44e9e79719 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:34.286204 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8c689e68-df97-4e4f-a445-ec44e9e79719 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:34.287001 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8c689e68-df97-4e4f-a445-ec44e9e79719 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:34.287704 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8c689e68-df97-4e4f-a445-ec44e9e79719 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:34.288314 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8c689e68-df97-4e4f-a445-ec44e9e79719 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:34.288914 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8c689e68-df97-4e4f-a445-ec44e9e79719 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:34.442457 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-8c689e68-df97-4e4f-a445-ec44e9e79719 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:34.453141 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-8c689e68-df97-4e4f-a445-ec44e9e79719 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254" status: 200 len: 1364 microversion: 2.1 time: 0.181732 Aug 08 02:30:34.453730 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 343/683] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:34 2022] GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254 => generated 1364 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:34.952934 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-5d7d9f83-b9fa-4169-b052-d649794c17f0 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:34.955051 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-5d7d9f83-b9fa-4169-b052-d649794c17f0 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: bd0f42d4-7cae-4736-8c36-fe5fa144908e] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:34.964801 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5d7d9f83-b9fa-4169-b052-d649794c17f0 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:34.965522 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5d7d9f83-b9fa-4169-b052-d649794c17f0 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:34.971738 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5d7d9f83-b9fa-4169-b052-d649794c17f0 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:34.976430 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5d7d9f83-b9fa-4169-b052-d649794c17f0 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:34.976430 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5d7d9f83-b9fa-4169-b052-d649794c17f0 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:34.976430 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5d7d9f83-b9fa-4169-b052-d649794c17f0 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:35.079071 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-8067c206-0f63-4192-9b13-c65e5d8a6eb7 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:35.081653 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-8067c206-0f63-4192-9b13-c65e5d8a6eb7 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: a5b1155c-5bba-46ae-8f3c-61f70c8a8a24] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:35.082800 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.objects.instance [None req-5d7d9f83-b9fa-4169-b052-d649794c17f0 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lazy-loading 'services' on Instance uuid bd0f42d4-7cae-4736-8c36-fe5fa144908e {{(pid=118351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 08 02:30:35.091087 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8067c206-0f63-4192-9b13-c65e5d8a6eb7 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:35.091665 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8067c206-0f63-4192-9b13-c65e5d8a6eb7 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:35.092176 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8067c206-0f63-4192-9b13-c65e5d8a6eb7 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:35.092967 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8067c206-0f63-4192-9b13-c65e5d8a6eb7 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:35.093429 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8067c206-0f63-4192-9b13-c65e5d8a6eb7 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:35.093855 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8067c206-0f63-4192-9b13-c65e5d8a6eb7 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:35.111237 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.wsgi [None req-8067c206-0f63-4192-9b13-c65e5d8a6eb7 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] HTTP exception thrown: Instance a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 could not be found. Aug 08 02:30:35.113119 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-8067c206-0f63-4192-9b13-c65e5d8a6eb7 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Request method failure captured: Aug 08 02:30:35.113119 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: request: GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 HTTP/1.1 Aug 08 02:30:35.113119 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept: application/json Aug 08 02:30:35.113119 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept-Encoding: identity Aug 08 02:30:35.113119 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Connection: close Aug 08 02:30:35.113119 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Length: 0 Aug 08 02:30:35.113119 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Type: application/json Aug 08 02:30:35.113119 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Host: 10.222.0.27 Aug 08 02:30:35.113119 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Openstack-System-Scope: None Aug 08 02:30:35.113119 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: User-Agent: python-urllib3/1.26.11 Aug 08 02:30:35.113119 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Auth-Token: gAAAAABi8HQXsGK5SLdeRHabROI8DgUVfT8-qt3OmjigKWIwr2r9ylfDom5hOf3PBwXXbVpQKaQQCg2GWwYCANhQNXkemxQtH2Z8vABNoiUbOWpRK-0iPQcLUey3hJs3m48mZ7E_Wt-sxx-vNwT0dxN6KL1UanLlX0GY-h9OUcxTwCxsePW3BgQ Aug 08 02:30:35.113119 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Id: None Aug 08 02:30:35.113119 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Name: None Aug 08 02:30:35.113119 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Identity-Status: Confirmed Aug 08 02:30:35.113119 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Is-Admin-Project: True Aug 08 02:30:35.113119 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Id: default Aug 08 02:30:35.113119 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Name: Default Aug 08 02:30:35.113119 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Id: 8af7d2bbc6d94273bd39c81423d8cebe Aug 08 02:30:35.113119 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-514896148 Aug 08 02:30:35.113119 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Role: reader,member Aug 08 02:30:35.113119 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Roles: reader,member Aug 08 02:30:35.114636 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/8af7d2bbc6d94273bd39c81423d8cebe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/8af7d2bbc6d94273bd39c81423d8cebe"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_8af7d2bbc6d94273bd39c81423d8cebe", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/8af7d2bbc6d94273bd39c81423d8cebe"}]}] Aug 08 02:30:35.114636 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant: tempest-ServerRescueNegativeTestJSON-514896148 Aug 08 02:30:35.114636 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Id: 8af7d2bbc6d94273bd39c81423d8cebe Aug 08 02:30:35.114636 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-514896148 Aug 08 02:30:35.114636 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User: tempest-ServerRescueNegativeTestJSON-514896148-project Aug 08 02:30:35.114636 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Id: default Aug 08 02:30:35.114636 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Name: Default Aug 08 02:30:35.114636 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Id: 5c8ae69712c14878bd2aab6d33512b7a Aug 08 02:30:35.114636 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Name: tempest-ServerRescueNegativeTestJSON-514896148-project Aug 08 02:30:35.114636 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: method: > Aug 08 02:30:35.114636 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: exception: Instance a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 could not be found. Aug 08 02:30:35.114636 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: action_args: {'id': 'a5b1155c-5bba-46ae-8f3c-61f70c8a8a24'} Aug 08 02:30:35.115500 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:30:35.115500 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:30:35.115500 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 08 02:30:35.115500 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 08 02:30:35.115500 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 08 02:30:35.115500 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 08 02:30:35.115500 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 08 02:30:35.115500 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 08 02:30:35.115500 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 08 02:30:35.115500 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 08 02:30:35.115500 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 could not be found. Aug 08 02:30:35.115500 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:30:35.115500 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:30:35.115500 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:30:35.115500 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:30:35.115500 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:30:35.115500 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:30:35.115500 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:30:35.115500 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:30:35.115500 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:30:35.116880 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:30:35.116880 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 08 02:30:35.116880 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 08 02:30:35.116880 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 08 02:30:35.116880 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 08 02:30:35.116880 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 08 02:30:35.116880 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:30:35.116880 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 could not be found. Aug 08 02:30:35.116880 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:30:35.116880 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:30:35.116880 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:30:35.116880 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:30:35.116880 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:30:35.116880 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:30:35.116880 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:30:35.116880 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:30:35.116880 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 could not be found. Aug 08 02:30:35.116880 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:30:35.118134 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-8067c206-0f63-4192-9b13-c65e5d8a6eb7 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Returning 404 to user: Instance a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 could not be found. {{(pid=118352) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:30:35.118134 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-8067c206-0f63-4192-9b13-c65e5d8a6eb7 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24" status: 404 len: 111 microversion: 2.1 time: 0.045070 Aug 08 02:30:35.118134 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 344/684] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:35 2022] GET /compute/v2.1/servers/a5b1155c-5bba-46ae-8f3c-61f70c8a8a24 => generated 111 bytes in 47 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Aug 08 02:30:35.124388 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-8c591f74-6d2a-43c2-b9be-a0e9505251b9 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '.version_select at 0x7efc740bc1f0>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:35.243016 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-8c591f74-6d2a-43c2-b9be-a0e9505251b9 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-2060462698" status: 202 len: 0 microversion: 2.1 time: 0.123219 Aug 08 02:30:35.249736 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 345/685] 10.222.0.27 () {58 vars in 1221 bytes} [Mon Aug 8 02:30:35 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-2060462698 => generated 0 bytes in 125 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 08 02:30:35.448950 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-5d7d9f83-b9fa-4169-b052-d649794c17f0 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "DELETE /compute/v2.1/servers/bd0f42d4-7cae-4736-8c36-fe5fa144908e/os-volume_attachments/17ffdefc-f349-47c8-b130-6d758fe1db67" status: 202 len: 0 microversion: 2.1 time: 0.499008 Aug 08 02:30:35.448950 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 341/686] 10.222.0.27 () {58 vars in 1416 bytes} [Mon Aug 8 02:30:34 2022] DELETE /compute/v2.1/servers/bd0f42d4-7cae-4736-8c36-fe5fa144908e/os-volume_attachments/17ffdefc-f349-47c8-b130-6d758fe1db67 => generated 0 bytes in 501 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 08 02:30:35.475866 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-902c8aca-4418-48fb-b924-59440cd9d5bc tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:35.483482 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-902c8aca-4418-48fb-b924-59440cd9d5bc tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: 48b3d4bf-6b20-4327-8f8d-521417405254] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:35.490699 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-902c8aca-4418-48fb-b924-59440cd9d5bc tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:35.491206 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-902c8aca-4418-48fb-b924-59440cd9d5bc tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:35.491651 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-902c8aca-4418-48fb-b924-59440cd9d5bc tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:35.493690 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-902c8aca-4418-48fb-b924-59440cd9d5bc tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:35.493690 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-902c8aca-4418-48fb-b924-59440cd9d5bc tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:35.493690 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-902c8aca-4418-48fb-b924-59440cd9d5bc tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:35.986946 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-902c8aca-4418-48fb-b924-59440cd9d5bc tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:35.997809 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-902c8aca-4418-48fb-b924-59440cd9d5bc tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254" status: 200 len: 1364 microversion: 2.1 time: 0.527431 Aug 08 02:30:35.998597 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 346/687] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:35 2022] GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254 => generated 1364 bytes in 529 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:36.102188 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [req-7317e4d1-b16f-4974-88e0-aba003a4d464 req-6cbf779d-95ab-46d2-b1a5-502852ff6a3a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "48b3d4bf-6b20-4327-8f8d-521417405254", "name": "network-vif-plugged", "status": "completed", "tag": "961375ea-c683-4138-905d-e3bf380590df"}]} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:30:36.118500 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-7317e4d1-b16f-4974-88e0-aba003a4d464 req-6cbf779d-95ab-46d2-b1a5-502852ff6a3a service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:36.119050 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-7317e4d1-b16f-4974-88e0-aba003a4d464 req-6cbf779d-95ab-46d2-b1a5-502852ff6a3a service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:36.119901 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-7317e4d1-b16f-4974-88e0-aba003a4d464 req-6cbf779d-95ab-46d2-b1a5-502852ff6a3a service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:36.133473 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.compute.server_external_events [req-7317e4d1-b16f-4974-88e0-aba003a4d464 req-6cbf779d-95ab-46d2-b1a5-502852ff6a3a service nova] Creating event network-vif-plugged:961375ea-c683-4138-905d-e3bf380590df for instance 48b3d4bf-6b20-4327-8f8d-521417405254 on hv-cinder-74014 Aug 08 02:30:36.139285 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [req-7317e4d1-b16f-4974-88e0-aba003a4d464 req-6cbf779d-95ab-46d2-b1a5-502852ff6a3a service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043369 Aug 08 02:30:36.140144 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 342/688] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:30:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:30:37.018992 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-5a941877-f662-4d0e-9f65-abd2675267f0 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:37.019820 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-5a941877-f662-4d0e-9f65-abd2675267f0 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: 48b3d4bf-6b20-4327-8f8d-521417405254] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:37.027046 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5a941877-f662-4d0e-9f65-abd2675267f0 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:37.027625 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5a941877-f662-4d0e-9f65-abd2675267f0 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:37.028156 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5a941877-f662-4d0e-9f65-abd2675267f0 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:37.028950 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5a941877-f662-4d0e-9f65-abd2675267f0 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:37.029480 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5a941877-f662-4d0e-9f65-abd2675267f0 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:37.029980 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5a941877-f662-4d0e-9f65-abd2675267f0 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:37.171767 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-5a941877-f662-4d0e-9f65-abd2675267f0 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:37.179745 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-5a941877-f662-4d0e-9f65-abd2675267f0 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254" status: 200 len: 1364 microversion: 2.1 time: 0.164581 Aug 08 02:30:37.180326 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 347/689] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:37 2022] GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254 => generated 1364 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:30:38.204558 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-91282595-6b1a-4d9e-aa72-e717e1ba03e0 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:38.205208 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-91282595-6b1a-4d9e-aa72-e717e1ba03e0 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: 48b3d4bf-6b20-4327-8f8d-521417405254] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:38.213995 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-91282595-6b1a-4d9e-aa72-e717e1ba03e0 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:38.214781 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-91282595-6b1a-4d9e-aa72-e717e1ba03e0 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:38.215342 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-91282595-6b1a-4d9e-aa72-e717e1ba03e0 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:38.216292 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-91282595-6b1a-4d9e-aa72-e717e1ba03e0 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:38.217116 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-91282595-6b1a-4d9e-aa72-e717e1ba03e0 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:38.217760 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-91282595-6b1a-4d9e-aa72-e717e1ba03e0 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:38.363396 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-91282595-6b1a-4d9e-aa72-e717e1ba03e0 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:38.375919 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-91282595-6b1a-4d9e-aa72-e717e1ba03e0 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254" status: 200 len: 1364 microversion: 2.1 time: 0.175809 Aug 08 02:30:38.376829 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 343/690] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:38 2022] GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254 => generated 1364 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:39.322316 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-cc68b212-7bb0-48de-9672-77883d421e3e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:39.323172 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-cc68b212-7bb0-48de-9672-77883d421e3e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:39.329784 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cc68b212-7bb0-48de-9672-77883d421e3e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:39.330261 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cc68b212-7bb0-48de-9672-77883d421e3e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:39.330752 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cc68b212-7bb0-48de-9672-77883d421e3e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:39.331438 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cc68b212-7bb0-48de-9672-77883d421e3e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:39.331886 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cc68b212-7bb0-48de-9672-77883d421e3e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:39.332310 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cc68b212-7bb0-48de-9672-77883d421e3e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:39.362882 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-cc68b212-7bb0-48de-9672-77883d421e3e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Going to try to terminate instance {{(pid=118352) delete /opt/stack/nova/nova/compute/api.py:2666}} Aug 08 02:30:39.398012 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-331ffe61-a287-439a-8744-e673085f183d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:39.400923 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-331ffe61-a287-439a-8744-e673085f183d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: 48b3d4bf-6b20-4327-8f8d-521417405254] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:39.408332 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-331ffe61-a287-439a-8744-e673085f183d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:39.408958 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-331ffe61-a287-439a-8744-e673085f183d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:39.409394 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-331ffe61-a287-439a-8744-e673085f183d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:39.410093 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-331ffe61-a287-439a-8744-e673085f183d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:39.410540 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-331ffe61-a287-439a-8744-e673085f183d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:39.418461 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-331ffe61-a287-439a-8744-e673085f183d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:39.629769 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-331ffe61-a287-439a-8744-e673085f183d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:39.629769 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-331ffe61-a287-439a-8744-e673085f183d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254" status: 200 len: 1550 microversion: 2.1 time: 0.166488 Aug 08 02:30:39.629769 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 344/691] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:39 2022] GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254 => generated 1550 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:30:39.638580 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-02292243-4e3c-463b-b20f-4d506f727151 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:39.639084 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-02292243-4e3c-463b-b20f-4d506f727151 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: 48b3d4bf-6b20-4327-8f8d-521417405254] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:39.647831 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-02292243-4e3c-463b-b20f-4d506f727151 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:39.648585 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-02292243-4e3c-463b-b20f-4d506f727151 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:39.649335 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-02292243-4e3c-463b-b20f-4d506f727151 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:39.650382 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-02292243-4e3c-463b-b20f-4d506f727151 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:39.652050 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-02292243-4e3c-463b-b20f-4d506f727151 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:39.652582 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-02292243-4e3c-463b-b20f-4d506f727151 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:39.754196 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-02292243-4e3c-463b-b20f-4d506f727151 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.119723 Aug 08 02:30:39.754804 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 345/692] 10.222.0.27 () {58 vars in 1275 bytes} [Mon Aug 8 02:30:39 2022] GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254/os-interface => generated 284 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 08 02:30:39.759708 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_api:SELECT=94,nova_api:DELETE=2,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:30:39.860489 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-cc68b212-7bb0-48de-9672-77883d421e3e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "DELETE /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 204 len: 0 microversion: 2.1 time: 0.542132 Aug 08 02:30:39.861911 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 348/693] 10.222.0.27 () {58 vars in 1239 bytes} [Mon Aug 8 02:30:39 2022] DELETE /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 08 02:30:39.871567 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-b9539ae0-3475-4ca3-ac02-fc6dc29260b2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:39.872179 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-b9539ae0-3475-4ca3-ac02-fc6dc29260b2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:39.878525 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b9539ae0-3475-4ca3-ac02-fc6dc29260b2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:39.879436 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b9539ae0-3475-4ca3-ac02-fc6dc29260b2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:39.880051 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b9539ae0-3475-4ca3-ac02-fc6dc29260b2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:39.882002 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b9539ae0-3475-4ca3-ac02-fc6dc29260b2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:39.882478 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b9539ae0-3475-4ca3-ac02-fc6dc29260b2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:39.882910 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b9539ae0-3475-4ca3-ac02-fc6dc29260b2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:40.043562 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-b9539ae0-3475-4ca3-ac02-fc6dc29260b2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:40.051467 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-b9539ae0-3475-4ca3-ac02-fc6dc29260b2 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1507 microversion: 2.1 time: 0.185651 Aug 08 02:30:40.052226 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 346/694] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:39 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1507 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:41.075275 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-e37b0812-2f3d-4ba0-9740-6bc4f33b6886 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:41.076722 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [req-6629bf64-75ca-454d-97a1-ed0ea110dc0c req-25f04d50-528d-45e6-96bb-7bbeec769a5f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "48b3d4bf-6b20-4327-8f8d-521417405254", "tag": "961375ea-c683-4138-905d-e3bf380590df"}]} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:30:41.078022 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-e37b0812-2f3d-4ba0-9740-6bc4f33b6886 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:41.087712 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-6629bf64-75ca-454d-97a1-ed0ea110dc0c req-25f04d50-528d-45e6-96bb-7bbeec769a5f service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:41.089842 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-6629bf64-75ca-454d-97a1-ed0ea110dc0c req-25f04d50-528d-45e6-96bb-7bbeec769a5f service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:41.090455 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-6629bf64-75ca-454d-97a1-ed0ea110dc0c req-25f04d50-528d-45e6-96bb-7bbeec769a5f service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:41.097489 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e37b0812-2f3d-4ba0-9740-6bc4f33b6886 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:41.098502 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e37b0812-2f3d-4ba0-9740-6bc4f33b6886 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:41.099776 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e37b0812-2f3d-4ba0-9740-6bc4f33b6886 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:41.101715 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e37b0812-2f3d-4ba0-9740-6bc4f33b6886 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:41.102994 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e37b0812-2f3d-4ba0-9740-6bc4f33b6886 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:41.104598 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e37b0812-2f3d-4ba0-9740-6bc4f33b6886 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:41.106765 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.compute.server_external_events [req-6629bf64-75ca-454d-97a1-ed0ea110dc0c req-25f04d50-528d-45e6-96bb-7bbeec769a5f service nova] Creating event network-changed:961375ea-c683-4138-905d-e3bf380590df for instance 48b3d4bf-6b20-4327-8f8d-521417405254 on hv-cinder-74014 Aug 08 02:30:41.113921 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [req-6629bf64-75ca-454d-97a1-ed0ea110dc0c req-25f04d50-528d-45e6-96bb-7bbeec769a5f service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041818 Aug 08 02:30:41.115359 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 347/695] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:30:41 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 08 02:30:41.271369 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-e37b0812-2f3d-4ba0-9740-6bc4f33b6886 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:41.280557 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-e37b0812-2f3d-4ba0-9740-6bc4f33b6886 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1507 microversion: 2.1 time: 0.212686 Aug 08 02:30:41.281308 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 349/696] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:41 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1507 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:42.303334 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-b7780a95-8e1e-47ab-b96c-a57a026bebd3 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:42.304033 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-b7780a95-8e1e-47ab-b96c-a57a026bebd3 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:42.311195 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b7780a95-8e1e-47ab-b96c-a57a026bebd3 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:42.311715 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b7780a95-8e1e-47ab-b96c-a57a026bebd3 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:42.312213 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b7780a95-8e1e-47ab-b96c-a57a026bebd3 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:42.312934 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b7780a95-8e1e-47ab-b96c-a57a026bebd3 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:42.313390 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b7780a95-8e1e-47ab-b96c-a57a026bebd3 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:42.313822 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b7780a95-8e1e-47ab-b96c-a57a026bebd3 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:42.457286 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-b7780a95-8e1e-47ab-b96c-a57a026bebd3 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:42.467340 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-b7780a95-8e1e-47ab-b96c-a57a026bebd3 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1507 microversion: 2.1 time: 0.167264 Aug 08 02:30:42.467931 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 348/697] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:42 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1507 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:43.490865 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-7ce66b25-2d02-4367-a5d8-af2e9f85300e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:43.495448 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [req-d32e62d5-9752-49d4-99e5-2744b8e5a372 req-ac3c440c-bc76-4cf5-bb25-c75bf5d166a9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "085a353c-1451-4abc-a9c5-dad9de93fcd4", "name": "network-vif-deleted", "tag": "f898d7da-2295-4c47-988e-433dec437c80"}]} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:30:43.497070 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-7ce66b25-2d02-4367-a5d8-af2e9f85300e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:43.510528 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7ce66b25-2d02-4367-a5d8-af2e9f85300e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:43.513203 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7ce66b25-2d02-4367-a5d8-af2e9f85300e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:43.516072 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-d32e62d5-9752-49d4-99e5-2744b8e5a372 req-ac3c440c-bc76-4cf5-bb25-c75bf5d166a9 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:43.516894 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7ce66b25-2d02-4367-a5d8-af2e9f85300e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:43.517862 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7ce66b25-2d02-4367-a5d8-af2e9f85300e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:43.518547 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7ce66b25-2d02-4367-a5d8-af2e9f85300e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:43.519207 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7ce66b25-2d02-4367-a5d8-af2e9f85300e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:43.519962 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-d32e62d5-9752-49d4-99e5-2744b8e5a372 req-ac3c440c-bc76-4cf5-bb25-c75bf5d166a9 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:43.525094 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-d32e62d5-9752-49d4-99e5-2744b8e5a372 req-ac3c440c-bc76-4cf5-bb25-c75bf5d166a9 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:43.549121 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.compute.server_external_events [req-d32e62d5-9752-49d4-99e5-2744b8e5a372 req-ac3c440c-bc76-4cf5-bb25-c75bf5d166a9 service nova] Creating event network-vif-deleted:f898d7da-2295-4c47-988e-433dec437c80 for instance 085a353c-1451-4abc-a9c5-dad9de93fcd4 on hv-cinder-74014 Aug 08 02:30:43.558402 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [req-d32e62d5-9752-49d4-99e5-2744b8e5a372 req-ac3c440c-bc76-4cf5-bb25-c75bf5d166a9 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.071758 Aug 08 02:30:43.561614 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 349/698] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:30:43 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:30:43.614426 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-7ce66b25-2d02-4367-a5d8-af2e9f85300e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:43.622671 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-7ce66b25-2d02-4367-a5d8-af2e9f85300e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 200 len: 1465 microversion: 2.1 time: 0.137953 Aug 08 02:30:43.623282 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 350/699] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:43 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 1465 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:43.822091 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-ec4f2c74-812a-4985-a231-793f676161c7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "09091e4c-3641-47bb-bdb7-14d386fcdcad"}} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:30:43.825676 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-ec4f2c74-812a-4985-a231-793f676161c7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: aa20d25c-bd00-41e6-b350-309368839578] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:43.832488 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ec4f2c74-812a-4985-a231-793f676161c7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:43.833567 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ec4f2c74-812a-4985-a231-793f676161c7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:43.834210 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ec4f2c74-812a-4985-a231-793f676161c7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:43.834915 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ec4f2c74-812a-4985-a231-793f676161c7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:43.835375 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ec4f2c74-812a-4985-a231-793f676161c7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:43.835815 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ec4f2c74-812a-4985-a231-793f676161c7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:43.932247 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.wsgi [None req-ec4f2c74-812a-4985-a231-793f676161c7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] HTTP exception thrown: Invalid volume: volume 09091e4c-3641-47bb-bdb7-14d386fcdcad is already attached to instances: aa20d25c-bd00-41e6-b350-309368839578 Aug 08 02:30:43.932247 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-ec4f2c74-812a-4985-a231-793f676161c7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Request method failure captured: Aug 08 02:30:43.932247 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: request: POST /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578/os-volume_attachments HTTP/1.1 Aug 08 02:30:43.932247 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Accept: application/json Aug 08 02:30:43.932247 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Accept-Encoding: identity Aug 08 02:30:43.932247 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Connection: close Aug 08 02:30:43.932247 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Content-Length: 74 Aug 08 02:30:43.932247 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Content-Type: application/json Aug 08 02:30:43.932247 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Host: 10.222.0.27 Aug 08 02:30:43.932247 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Openstack-System-Scope: None Aug 08 02:30:43.932247 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: User-Agent: python-urllib3/1.26.11 Aug 08 02:30:43.932247 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Auth-Token: gAAAAABi8HQUs7QI_wjv0vSDVlijMnZ9-Lkgqc--Bu80jwWL-CoTBjbpe1sae_INrRIGeFqgwK-ezXnEOgqpG2NO5P97SEgPcY4zY_QgnaTA6xv-rzef5kDF5s4pU5M8QdM-T8eUF020_v3zsmgR78rxXaglg0C5QuxYvT8R4THTJZuwa_f26Bo Aug 08 02:30:43.932247 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Domain-Id: None Aug 08 02:30:43.932247 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Domain-Name: None Aug 08 02:30:43.932247 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Identity-Status: Confirmed Aug 08 02:30:43.932247 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Is-Admin-Project: True Aug 08 02:30:43.932247 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Domain-Id: default Aug 08 02:30:43.932247 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Domain-Name: Default Aug 08 02:30:43.932247 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Id: ca29b2b9a59d4826bfd25f4dcfa9cffe Aug 08 02:30:43.932247 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Name: tempest-AttachVolumeNegativeTest-420450256 Aug 08 02:30:43.932247 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Role: reader,member Aug 08 02:30:43.932247 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Roles: reader,member Aug 08 02:30:43.933680 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/ca29b2b9a59d4826bfd25f4dcfa9cffe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/ca29b2b9a59d4826bfd25f4dcfa9cffe"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_ca29b2b9a59d4826bfd25f4dcfa9cffe", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/ca29b2b9a59d4826bfd25f4dcfa9cffe"}]}] Aug 08 02:30:43.933680 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant: tempest-AttachVolumeNegativeTest-420450256 Aug 08 02:30:43.933680 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant-Id: ca29b2b9a59d4826bfd25f4dcfa9cffe Aug 08 02:30:43.933680 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-420450256 Aug 08 02:30:43.933680 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User: tempest-AttachVolumeNegativeTest-420450256-project Aug 08 02:30:43.933680 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Domain-Id: default Aug 08 02:30:43.933680 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Domain-Name: Default Aug 08 02:30:43.933680 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Id: 578cd837d9fe442587f8c6c8528334c0 Aug 08 02:30:43.933680 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Name: tempest-AttachVolumeNegativeTest-420450256-project Aug 08 02:30:43.933680 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Aug 08 02:30:43.933680 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: {"volumeAttachment": {"volumeId": "09091e4c-3641-47bb-bdb7-14d386fcdcad"}} Aug 08 02:30:43.933680 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: method: > Aug 08 02:30:43.934435 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: exception: Invalid volume: volume 09091e4c-3641-47bb-bdb7-14d386fcdcad is already attached to instances: aa20d25c-bd00-41e6-b350-309368839578 Aug 08 02:30:43.934435 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: action_args: {'server_id': 'aa20d25c-bd00-41e6-b350-309368839578', 'body': {'volumeAttachment': {'volumeId': '09091e4c-3641-47bb-bdb7-14d386fcdcad'}}} Aug 08 02:30:43.934435 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:30:43.934435 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:30:43.934435 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Aug 08 02:30:43.934435 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Aug 08 02:30:43.934435 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 08 02:30:43.934435 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 08 02:30:43.934435 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Aug 08 02:30:43.934435 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 08 02:30:43.934435 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5066, in attach_volume Aug 08 02:30:43.934435 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Aug 08 02:30:43.934435 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4906, in _check_volume_already_attached Aug 08 02:30:43.934435 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Aug 08 02:30:43.934435 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume 09091e4c-3641-47bb-bdb7-14d386fcdcad is already attached to instances: aa20d25c-bd00-41e6-b350-309368839578 Aug 08 02:30:43.934435 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:30:43.934435 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:30:43.934435 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:30:43.934435 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:30:43.935910 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:30:43.935910 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:30:43.935910 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:30:43.935910 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:30:43.935910 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:30:43.935910 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:30:43.935910 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 08 02:30:43.935910 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 08 02:30:43.935910 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 08 02:30:43.935910 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 08 02:30:43.935910 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 08 02:30:43.935910 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 08 02:30:43.935910 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Aug 08 02:30:43.935910 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 08 02:30:43.935910 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume 09091e4c-3641-47bb-bdb7-14d386fcdcad is already attached to instances: aa20d25c-bd00-41e6-b350-309368839578 Aug 08 02:30:43.935910 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:30:43.935910 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:30:43.935910 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:30:43.935910 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:30:43.935910 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:30:43.937688 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:30:43.937688 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:30:43.937688 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:30:43.937688 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume 09091e4c-3641-47bb-bdb7-14d386fcdcad is already attached to instances: aa20d25c-bd00-41e6-b350-309368839578 Aug 08 02:30:43.937688 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:30:43.937688 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-ec4f2c74-812a-4985-a231-793f676161c7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Returning 400 to user: Invalid volume: volume 09091e4c-3641-47bb-bdb7-14d386fcdcad is already attached to instances: aa20d25c-bd00-41e6-b350-309368839578 {{(pid=118351) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:30:43.937688 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-ec4f2c74-812a-4985-a231-793f676161c7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "POST /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.113804 Aug 08 02:30:43.937688 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 350/700] 10.222.0.27 () {60 vars in 1323 bytes} [Mon Aug 8 02:30:43 2022] POST /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578/os-volume_attachments => generated 174 bytes in 116 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Aug 08 02:30:43.997205 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-945d4b48-c7c6-40c3-875d-04a7d28a4f9d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:44.006695 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-945d4b48-c7c6-40c3-875d-04a7d28a4f9d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: aa20d25c-bd00-41e6-b350-309368839578] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:44.006695 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-945d4b48-c7c6-40c3-875d-04a7d28a4f9d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:44.006695 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-945d4b48-c7c6-40c3-875d-04a7d28a4f9d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:44.006695 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-945d4b48-c7c6-40c3-875d-04a7d28a4f9d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:44.007079 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-945d4b48-c7c6-40c3-875d-04a7d28a4f9d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:44.007079 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-945d4b48-c7c6-40c3-875d-04a7d28a4f9d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:44.007079 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-945d4b48-c7c6-40c3-875d-04a7d28a4f9d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:44.024387 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:30:44.089986 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.objects.instance [None req-945d4b48-c7c6-40c3-875d-04a7d28a4f9d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lazy-loading 'services' on Instance uuid aa20d25c-bd00-41e6-b350-309368839578 {{(pid=118352) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 08 02:30:44.341298 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-945d4b48-c7c6-40c3-875d-04a7d28a4f9d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "DELETE /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578/os-volume_attachments/09091e4c-3641-47bb-bdb7-14d386fcdcad" status: 202 len: 0 microversion: 2.1 time: 0.348644 Aug 08 02:30:44.342097 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 351/701] 10.222.0.27 () {58 vars in 1416 bytes} [Mon Aug 8 02:30:43 2022] DELETE /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578/os-volume_attachments/09091e4c-3641-47bb-bdb7-14d386fcdcad => generated 0 bytes in 351 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 08 02:30:44.351135 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-f15ec67f-b59c-4997-89b9-df7aac7b79bd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:44.353300 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-f15ec67f-b59c-4997-89b9-df7aac7b79bd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: aa20d25c-bd00-41e6-b350-309368839578] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:44.361628 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f15ec67f-b59c-4997-89b9-df7aac7b79bd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:44.362115 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f15ec67f-b59c-4997-89b9-df7aac7b79bd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:44.362549 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f15ec67f-b59c-4997-89b9-df7aac7b79bd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:44.363244 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f15ec67f-b59c-4997-89b9-df7aac7b79bd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:44.363769 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f15ec67f-b59c-4997-89b9-df7aac7b79bd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:44.364194 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f15ec67f-b59c-4997-89b9-df7aac7b79bd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:44.393631 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-f15ec67f-b59c-4997-89b9-df7aac7b79bd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047158 Aug 08 02:30:44.394265 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 351/702] 10.222.0.27 () {58 vars in 1302 bytes} [Mon Aug 8 02:30:44 2022] GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:30:44.644817 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-2257aa2e-9492-4425-a268-e4bac1df7a35 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:44.645557 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-2257aa2e-9492-4425-a268-e4bac1df7a35 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: 085a353c-1451-4abc-a9c5-dad9de93fcd4] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:44.652652 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2257aa2e-9492-4425-a268-e4bac1df7a35 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:44.653161 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2257aa2e-9492-4425-a268-e4bac1df7a35 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:44.653632 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2257aa2e-9492-4425-a268-e4bac1df7a35 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:44.654322 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2257aa2e-9492-4425-a268-e4bac1df7a35 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:44.654778 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2257aa2e-9492-4425-a268-e4bac1df7a35 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:44.655205 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2257aa2e-9492-4425-a268-e4bac1df7a35 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:44.668504 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.wsgi [None req-2257aa2e-9492-4425-a268-e4bac1df7a35 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] HTTP exception thrown: Instance 085a353c-1451-4abc-a9c5-dad9de93fcd4 could not be found. Aug 08 02:30:44.670178 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-2257aa2e-9492-4425-a268-e4bac1df7a35 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Request method failure captured: Aug 08 02:30:44.670178 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: request: GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 HTTP/1.1 Aug 08 02:30:44.670178 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept: application/json Aug 08 02:30:44.670178 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept-Encoding: identity Aug 08 02:30:44.670178 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Connection: close Aug 08 02:30:44.670178 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Length: 0 Aug 08 02:30:44.670178 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Type: application/json Aug 08 02:30:44.670178 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Host: 10.222.0.27 Aug 08 02:30:44.670178 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Openstack-System-Scope: None Aug 08 02:30:44.670178 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: User-Agent: python-urllib3/1.26.11 Aug 08 02:30:44.670178 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Auth-Token: gAAAAABi8HQXsGK5SLdeRHabROI8DgUVfT8-qt3OmjigKWIwr2r9ylfDom5hOf3PBwXXbVpQKaQQCg2GWwYCANhQNXkemxQtH2Z8vABNoiUbOWpRK-0iPQcLUey3hJs3m48mZ7E_Wt-sxx-vNwT0dxN6KL1UanLlX0GY-h9OUcxTwCxsePW3BgQ Aug 08 02:30:44.670178 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Id: None Aug 08 02:30:44.670178 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Name: None Aug 08 02:30:44.670178 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Identity-Status: Confirmed Aug 08 02:30:44.670178 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Is-Admin-Project: True Aug 08 02:30:44.670178 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Id: default Aug 08 02:30:44.670178 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Name: Default Aug 08 02:30:44.670178 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Id: 8af7d2bbc6d94273bd39c81423d8cebe Aug 08 02:30:44.670178 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-514896148 Aug 08 02:30:44.670178 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Role: reader,member Aug 08 02:30:44.670178 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Roles: reader,member Aug 08 02:30:44.671770 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/8af7d2bbc6d94273bd39c81423d8cebe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/8af7d2bbc6d94273bd39c81423d8cebe"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_8af7d2bbc6d94273bd39c81423d8cebe", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/8af7d2bbc6d94273bd39c81423d8cebe"}]}] Aug 08 02:30:44.671770 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant: tempest-ServerRescueNegativeTestJSON-514896148 Aug 08 02:30:44.671770 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Id: 8af7d2bbc6d94273bd39c81423d8cebe Aug 08 02:30:44.671770 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-514896148 Aug 08 02:30:44.671770 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User: tempest-ServerRescueNegativeTestJSON-514896148-project Aug 08 02:30:44.671770 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Id: default Aug 08 02:30:44.671770 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Name: Default Aug 08 02:30:44.671770 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Id: 5c8ae69712c14878bd2aab6d33512b7a Aug 08 02:30:44.671770 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Name: tempest-ServerRescueNegativeTestJSON-514896148-project Aug 08 02:30:44.671770 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: method: > Aug 08 02:30:44.671770 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: exception: Instance 085a353c-1451-4abc-a9c5-dad9de93fcd4 could not be found. Aug 08 02:30:44.671770 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: action_args: {'id': '085a353c-1451-4abc-a9c5-dad9de93fcd4'} Aug 08 02:30:44.672712 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:30:44.672712 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:30:44.672712 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 08 02:30:44.672712 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 08 02:30:44.672712 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 08 02:30:44.672712 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 08 02:30:44.672712 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 08 02:30:44.672712 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 08 02:30:44.672712 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 08 02:30:44.672712 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 08 02:30:44.672712 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 085a353c-1451-4abc-a9c5-dad9de93fcd4 could not be found. Aug 08 02:30:44.672712 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:30:44.672712 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:30:44.672712 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:30:44.672712 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:30:44.672712 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:30:44.672712 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:30:44.672712 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:30:44.672712 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:30:44.672712 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:30:44.674244 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:30:44.674244 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 08 02:30:44.674244 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 08 02:30:44.674244 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 08 02:30:44.674244 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 08 02:30:44.674244 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 08 02:30:44.674244 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:30:44.674244 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 085a353c-1451-4abc-a9c5-dad9de93fcd4 could not be found. Aug 08 02:30:44.674244 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:30:44.674244 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:30:44.674244 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:30:44.674244 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:30:44.674244 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:30:44.674244 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:30:44.674244 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:30:44.674244 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:30:44.674244 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 085a353c-1451-4abc-a9c5-dad9de93fcd4 could not be found. Aug 08 02:30:44.674244 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:30:44.675745 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-2257aa2e-9492-4425-a268-e4bac1df7a35 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Returning 404 to user: Instance 085a353c-1451-4abc-a9c5-dad9de93fcd4 could not be found. {{(pid=118352) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:30:44.675745 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-2257aa2e-9492-4425-a268-e4bac1df7a35 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4" status: 404 len: 111 microversion: 2.1 time: 0.031513 Aug 08 02:30:44.675745 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 352/703] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:44 2022] GET /compute/v2.1/servers/085a353c-1451-4abc-a9c5-dad9de93fcd4 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 08 02:30:44.681096 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-133c18b6-8b3c-4c40-939a-68fb54fd5c6f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:44.687594 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-133c18b6-8b3c-4c40-939a-68fb54fd5c6f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: ed75f58f-9135-4d97-b24a-2f8fcdab53ce] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:44.687937 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-133c18b6-8b3c-4c40-939a-68fb54fd5c6f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:44.688424 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-133c18b6-8b3c-4c40-939a-68fb54fd5c6f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:44.688921 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-133c18b6-8b3c-4c40-939a-68fb54fd5c6f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:44.689615 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-133c18b6-8b3c-4c40-939a-68fb54fd5c6f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:44.690075 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-133c18b6-8b3c-4c40-939a-68fb54fd5c6f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:44.690502 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-133c18b6-8b3c-4c40-939a-68fb54fd5c6f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:44.716721 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-133c18b6-8b3c-4c40-939a-68fb54fd5c6f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: ed75f58f-9135-4d97-b24a-2f8fcdab53ce] Going to try to terminate instance {{(pid=118351) delete /opt/stack/nova/nova/compute/api.py:2666}} Aug 08 02:30:44.819612 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:30:44.979462 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-133c18b6-8b3c-4c40-939a-68fb54fd5c6f tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "DELETE /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce" status: 204 len: 0 microversion: 2.1 time: 0.301900 Aug 08 02:30:44.980187 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 352/704] 10.222.0.27 () {58 vars in 1239 bytes} [Mon Aug 8 02:30:44 2022] DELETE /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce => generated 0 bytes in 304 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 08 02:30:44.988056 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-67b8ef9b-8864-4b0b-8ef9-75dca4ae65be tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:44.988902 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-67b8ef9b-8864-4b0b-8ef9-75dca4ae65be tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: ed75f58f-9135-4d97-b24a-2f8fcdab53ce] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:44.995461 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-67b8ef9b-8864-4b0b-8ef9-75dca4ae65be tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:44.997363 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-67b8ef9b-8864-4b0b-8ef9-75dca4ae65be tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:44.997363 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-67b8ef9b-8864-4b0b-8ef9-75dca4ae65be tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:44.997363 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-67b8ef9b-8864-4b0b-8ef9-75dca4ae65be tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:44.997652 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-67b8ef9b-8864-4b0b-8ef9-75dca4ae65be tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:44.997890 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-67b8ef9b-8864-4b0b-8ef9-75dca4ae65be tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:45.114347 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-9566951a-dd6b-431c-b665-cbd1bc3529c9 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:45.117937 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-9566951a-dd6b-431c-b665-cbd1bc3529c9 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: bd0f42d4-7cae-4736-8c36-fe5fa144908e] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:45.122301 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-9566951a-dd6b-431c-b665-cbd1bc3529c9 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:45.122863 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-9566951a-dd6b-431c-b665-cbd1bc3529c9 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:45.123306 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-9566951a-dd6b-431c-b665-cbd1bc3529c9 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:45.124034 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-9566951a-dd6b-431c-b665-cbd1bc3529c9 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:45.124498 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-9566951a-dd6b-431c-b665-cbd1bc3529c9 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:45.124962 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-9566951a-dd6b-431c-b665-cbd1bc3529c9 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:45.134437 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-67b8ef9b-8864-4b0b-8ef9-75dca4ae65be tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:45.142810 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-67b8ef9b-8864-4b0b-8ef9-75dca4ae65be tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce" status: 200 len: 1522 microversion: 2.1 time: 0.158481 Aug 08 02:30:45.158094 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 353/705] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:44 2022] GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce => generated 1522 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:45.158236 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-9566951a-dd6b-431c-b665-cbd1bc3529c9 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: bd0f42d4-7cae-4736-8c36-fe5fa144908e] Going to try to terminate instance {{(pid=118351) delete /opt/stack/nova/nova/compute/api.py:2666}} Aug 08 02:30:45.406382 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-2d9d1e2a-3966-4b30-b25b-c21ab02076ad tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:45.408621 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-2d9d1e2a-3966-4b30-b25b-c21ab02076ad tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: aa20d25c-bd00-41e6-b350-309368839578] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:45.414615 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2d9d1e2a-3966-4b30-b25b-c21ab02076ad tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:45.415118 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2d9d1e2a-3966-4b30-b25b-c21ab02076ad tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:45.415525 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2d9d1e2a-3966-4b30-b25b-c21ab02076ad tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:45.417229 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2d9d1e2a-3966-4b30-b25b-c21ab02076ad tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:45.417368 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2d9d1e2a-3966-4b30-b25b-c21ab02076ad tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:45.417795 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2d9d1e2a-3966-4b30-b25b-c21ab02076ad tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:45.447136 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-2d9d1e2a-3966-4b30-b25b-c21ab02076ad tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044863 Aug 08 02:30:45.447686 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 354/706] 10.222.0.27 () {58 vars in 1302 bytes} [Mon Aug 8 02:30:45 2022] GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 08 02:30:45.539659 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-9566951a-dd6b-431c-b665-cbd1bc3529c9 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "DELETE /compute/v2.1/servers/bd0f42d4-7cae-4736-8c36-fe5fa144908e" status: 204 len: 0 microversion: 2.1 time: 0.429618 Aug 08 02:30:45.540249 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 353/707] 10.222.0.27 () {58 vars in 1239 bytes} [Mon Aug 8 02:30:45 2022] DELETE /compute/v2.1/servers/bd0f42d4-7cae-4736-8c36-fe5fa144908e => generated 0 bytes in 431 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Aug 08 02:30:45.548930 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-b83230c8-edbf-4d1f-87d1-27cdf3f12935 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:45.549561 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-b83230c8-edbf-4d1f-87d1-27cdf3f12935 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: bd0f42d4-7cae-4736-8c36-fe5fa144908e] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:45.555500 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b83230c8-edbf-4d1f-87d1-27cdf3f12935 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:45.555981 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b83230c8-edbf-4d1f-87d1-27cdf3f12935 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:45.556413 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b83230c8-edbf-4d1f-87d1-27cdf3f12935 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:45.557123 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b83230c8-edbf-4d1f-87d1-27cdf3f12935 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:45.557715 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b83230c8-edbf-4d1f-87d1-27cdf3f12935 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:45.558137 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b83230c8-edbf-4d1f-87d1-27cdf3f12935 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:45.687721 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-b83230c8-edbf-4d1f-87d1-27cdf3f12935 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5e3c891f2f3148f4aaba41a14d507cdf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7203c52ae1f946bd9717a554316e4f4c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:45.695357 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-b83230c8-edbf-4d1f-87d1-27cdf3f12935 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "GET /compute/v2.1/servers/bd0f42d4-7cae-4736-8c36-fe5fa144908e" status: 200 len: 1514 microversion: 2.1 time: 0.150306 Aug 08 02:30:45.695916 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 355/708] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:45 2022] GET /compute/v2.1/servers/bd0f42d4-7cae-4736-8c36-fe5fa144908e => generated 1514 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:46.166665 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-17e003c4-32aa-4b19-8d00-e8b61abb506e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:46.167359 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-17e003c4-32aa-4b19-8d00-e8b61abb506e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: ed75f58f-9135-4d97-b24a-2f8fcdab53ce] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:46.173739 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-17e003c4-32aa-4b19-8d00-e8b61abb506e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:46.174213 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-17e003c4-32aa-4b19-8d00-e8b61abb506e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:46.176294 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-17e003c4-32aa-4b19-8d00-e8b61abb506e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:46.176294 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-17e003c4-32aa-4b19-8d00-e8b61abb506e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:46.176294 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-17e003c4-32aa-4b19-8d00-e8b61abb506e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:46.176294 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-17e003c4-32aa-4b19-8d00-e8b61abb506e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:46.308254 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-17e003c4-32aa-4b19-8d00-e8b61abb506e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:46.315896 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-17e003c4-32aa-4b19-8d00-e8b61abb506e tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce" status: 200 len: 1522 microversion: 2.1 time: 0.153894 Aug 08 02:30:46.316555 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 354/709] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:46 2022] GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce => generated 1522 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:46.459536 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-a6e33e01-9774-4dd2-aaad-aee268dd2855 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:46.461759 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-a6e33e01-9774-4dd2-aaad-aee268dd2855 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: aa20d25c-bd00-41e6-b350-309368839578] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:46.468114 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a6e33e01-9774-4dd2-aaad-aee268dd2855 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:46.468592 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a6e33e01-9774-4dd2-aaad-aee268dd2855 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:46.469057 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a6e33e01-9774-4dd2-aaad-aee268dd2855 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:46.469734 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a6e33e01-9774-4dd2-aaad-aee268dd2855 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:46.470183 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a6e33e01-9774-4dd2-aaad-aee268dd2855 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:46.470605 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a6e33e01-9774-4dd2-aaad-aee268dd2855 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:46.499622 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-a6e33e01-9774-4dd2-aaad-aee268dd2855 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044006 Aug 08 02:30:46.500169 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 356/710] 10.222.0.27 () {58 vars in 1302 bytes} [Mon Aug 8 02:30:46 2022] GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:30:46.716799 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-4a2ab7cc-5534-4233-b1e3-f4e772961d5f tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:46.717402 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-4a2ab7cc-5534-4233-b1e3-f4e772961d5f tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: bd0f42d4-7cae-4736-8c36-fe5fa144908e] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:46.723538 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4a2ab7cc-5534-4233-b1e3-f4e772961d5f tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:46.723974 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4a2ab7cc-5534-4233-b1e3-f4e772961d5f tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:46.724400 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4a2ab7cc-5534-4233-b1e3-f4e772961d5f tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:46.725111 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4a2ab7cc-5534-4233-b1e3-f4e772961d5f tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:46.725561 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4a2ab7cc-5534-4233-b1e3-f4e772961d5f tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:46.725979 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4a2ab7cc-5534-4233-b1e3-f4e772961d5f tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:46.860638 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-4a2ab7cc-5534-4233-b1e3-f4e772961d5f tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5e3c891f2f3148f4aaba41a14d507cdf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7203c52ae1f946bd9717a554316e4f4c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:46.867994 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-4a2ab7cc-5534-4233-b1e3-f4e772961d5f tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "GET /compute/v2.1/servers/bd0f42d4-7cae-4736-8c36-fe5fa144908e" status: 200 len: 1514 microversion: 2.1 time: 0.155278 Aug 08 02:30:46.868689 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 355/711] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:46 2022] GET /compute/v2.1/servers/bd0f42d4-7cae-4736-8c36-fe5fa144908e => generated 1514 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:47.340146 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-583668ea-4907-4dea-8090-f00cda9f23f5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:47.340858 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-583668ea-4907-4dea-8090-f00cda9f23f5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: ed75f58f-9135-4d97-b24a-2f8fcdab53ce] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:47.346803 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-583668ea-4907-4dea-8090-f00cda9f23f5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:47.347283 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-583668ea-4907-4dea-8090-f00cda9f23f5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:47.347724 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-583668ea-4907-4dea-8090-f00cda9f23f5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:47.348404 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-583668ea-4907-4dea-8090-f00cda9f23f5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:47.348895 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-583668ea-4907-4dea-8090-f00cda9f23f5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:47.349327 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-583668ea-4907-4dea-8090-f00cda9f23f5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:47.477724 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-583668ea-4907-4dea-8090-f00cda9f23f5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:47.485326 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-583668ea-4907-4dea-8090-f00cda9f23f5 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce" status: 200 len: 1522 microversion: 2.1 time: 0.151131 Aug 08 02:30:47.485962 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 357/712] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:47 2022] GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce => generated 1522 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:47.511743 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-ec0db84c-9984-4a45-8bd1-2f8a65152e84 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:47.513986 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-ec0db84c-9984-4a45-8bd1-2f8a65152e84 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: aa20d25c-bd00-41e6-b350-309368839578] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:47.520269 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ec0db84c-9984-4a45-8bd1-2f8a65152e84 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:47.520793 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ec0db84c-9984-4a45-8bd1-2f8a65152e84 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:47.521236 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ec0db84c-9984-4a45-8bd1-2f8a65152e84 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:47.521921 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ec0db84c-9984-4a45-8bd1-2f8a65152e84 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:47.522374 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ec0db84c-9984-4a45-8bd1-2f8a65152e84 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:47.522795 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ec0db84c-9984-4a45-8bd1-2f8a65152e84 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:47.551699 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-ec0db84c-9984-4a45-8bd1-2f8a65152e84 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044335 Aug 08 02:30:47.552273 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 356/713] 10.222.0.27 () {58 vars in 1302 bytes} [Mon Aug 8 02:30:47 2022] GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:30:47.890360 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-a5ae9e37-0018-4feb-ae31-f99690d6124d tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:47.890972 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-a5ae9e37-0018-4feb-ae31-f99690d6124d tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: bd0f42d4-7cae-4736-8c36-fe5fa144908e] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:47.896949 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a5ae9e37-0018-4feb-ae31-f99690d6124d tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:47.897434 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a5ae9e37-0018-4feb-ae31-f99690d6124d tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:47.897872 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a5ae9e37-0018-4feb-ae31-f99690d6124d tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:47.898559 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a5ae9e37-0018-4feb-ae31-f99690d6124d tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:47.899013 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a5ae9e37-0018-4feb-ae31-f99690d6124d tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:47.899442 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a5ae9e37-0018-4feb-ae31-f99690d6124d tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:48.030429 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-a5ae9e37-0018-4feb-ae31-f99690d6124d tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5e3c891f2f3148f4aaba41a14d507cdf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7203c52ae1f946bd9717a554316e4f4c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:48.037904 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-a5ae9e37-0018-4feb-ae31-f99690d6124d tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "GET /compute/v2.1/servers/bd0f42d4-7cae-4736-8c36-fe5fa144908e" status: 200 len: 1514 microversion: 2.1 time: 0.151418 Aug 08 02:30:48.038461 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 358/714] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:47 2022] GET /compute/v2.1/servers/bd0f42d4-7cae-4736-8c36-fe5fa144908e => generated 1514 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:48.506116 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-293f0399-e521-4f18-b5d4-c4e24fe2f6ea tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:48.507185 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-293f0399-e521-4f18-b5d4-c4e24fe2f6ea tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: ed75f58f-9135-4d97-b24a-2f8fcdab53ce] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:48.512906 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-293f0399-e521-4f18-b5d4-c4e24fe2f6ea tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:48.513483 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-293f0399-e521-4f18-b5d4-c4e24fe2f6ea tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:48.513798 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-293f0399-e521-4f18-b5d4-c4e24fe2f6ea tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:48.514519 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-293f0399-e521-4f18-b5d4-c4e24fe2f6ea tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:48.514970 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-293f0399-e521-4f18-b5d4-c4e24fe2f6ea tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:48.515390 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-293f0399-e521-4f18-b5d4-c4e24fe2f6ea tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:48.564527 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-5b5e1fb0-6548-4ba3-ad0e-845a4b0bbfee tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:48.566800 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-5b5e1fb0-6548-4ba3-ad0e-845a4b0bbfee tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: aa20d25c-bd00-41e6-b350-309368839578] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:48.573618 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5b5e1fb0-6548-4ba3-ad0e-845a4b0bbfee tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:48.575821 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5b5e1fb0-6548-4ba3-ad0e-845a4b0bbfee tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:48.575821 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5b5e1fb0-6548-4ba3-ad0e-845a4b0bbfee tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:48.575821 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5b5e1fb0-6548-4ba3-ad0e-845a4b0bbfee tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:48.575821 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5b5e1fb0-6548-4ba3-ad0e-845a4b0bbfee tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:48.576215 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5b5e1fb0-6548-4ba3-ad0e-845a4b0bbfee tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:48.610580 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-5b5e1fb0-6548-4ba3-ad0e-845a4b0bbfee tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050864 Aug 08 02:30:48.611159 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 359/715] 10.222.0.27 () {58 vars in 1302 bytes} [Mon Aug 8 02:30:48 2022] GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:30:48.658582 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-293f0399-e521-4f18-b5d4-c4e24fe2f6ea tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:48.666953 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-293f0399-e521-4f18-b5d4-c4e24fe2f6ea tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce" status: 200 len: 1522 microversion: 2.1 time: 0.164036 Aug 08 02:30:48.667612 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 357/716] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:48 2022] GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce => generated 1522 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:49.060092 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-c64bb3b4-29fd-4326-a07e-dbf88f6d598b tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:49.060740 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-c64bb3b4-29fd-4326-a07e-dbf88f6d598b tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: bd0f42d4-7cae-4736-8c36-fe5fa144908e] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:49.066675 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c64bb3b4-29fd-4326-a07e-dbf88f6d598b tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:49.067168 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c64bb3b4-29fd-4326-a07e-dbf88f6d598b tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:49.067609 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c64bb3b4-29fd-4326-a07e-dbf88f6d598b tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:49.068293 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c64bb3b4-29fd-4326-a07e-dbf88f6d598b tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:49.068783 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c64bb3b4-29fd-4326-a07e-dbf88f6d598b tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:49.069480 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c64bb3b4-29fd-4326-a07e-dbf88f6d598b tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:49.198854 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-c64bb3b4-29fd-4326-a07e-dbf88f6d598b tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5e3c891f2f3148f4aaba41a14d507cdf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7203c52ae1f946bd9717a554316e4f4c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:49.206364 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-c64bb3b4-29fd-4326-a07e-dbf88f6d598b tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "GET /compute/v2.1/servers/bd0f42d4-7cae-4736-8c36-fe5fa144908e" status: 200 len: 1514 microversion: 2.1 time: 0.150103 Aug 08 02:30:49.206928 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 360/717] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:49 2022] GET /compute/v2.1/servers/bd0f42d4-7cae-4736-8c36-fe5fa144908e => generated 1514 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:49.623286 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-0365b306-de41-4786-9874-4a2495f3723d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:49.625510 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-0365b306-de41-4786-9874-4a2495f3723d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: aa20d25c-bd00-41e6-b350-309368839578] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:49.631922 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0365b306-de41-4786-9874-4a2495f3723d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:49.632486 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0365b306-de41-4786-9874-4a2495f3723d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:49.632953 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0365b306-de41-4786-9874-4a2495f3723d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:49.633639 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0365b306-de41-4786-9874-4a2495f3723d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:49.634098 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0365b306-de41-4786-9874-4a2495f3723d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:49.634741 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0365b306-de41-4786-9874-4a2495f3723d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:49.665112 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-0365b306-de41-4786-9874-4a2495f3723d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046224 Aug 08 02:30:49.665674 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 358/718] 10.222.0.27 () {58 vars in 1302 bytes} [Mon Aug 8 02:30:49 2022] GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:30:49.688053 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-6e48f5ce-c4de-44d8-b9f8-9247e5ff142a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:49.688710 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-6e48f5ce-c4de-44d8-b9f8-9247e5ff142a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: ed75f58f-9135-4d97-b24a-2f8fcdab53ce] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:49.695485 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6e48f5ce-c4de-44d8-b9f8-9247e5ff142a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:49.695968 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6e48f5ce-c4de-44d8-b9f8-9247e5ff142a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:49.696405 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6e48f5ce-c4de-44d8-b9f8-9247e5ff142a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:49.697113 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6e48f5ce-c4de-44d8-b9f8-9247e5ff142a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:49.697885 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6e48f5ce-c4de-44d8-b9f8-9247e5ff142a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:49.698121 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6e48f5ce-c4de-44d8-b9f8-9247e5ff142a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:49.829581 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-6e48f5ce-c4de-44d8-b9f8-9247e5ff142a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:49.837284 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-6e48f5ce-c4de-44d8-b9f8-9247e5ff142a tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce" status: 200 len: 1522 microversion: 2.1 time: 0.153050 Aug 08 02:30:49.837790 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 361/719] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:49 2022] GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce => generated 1522 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:30:50.227376 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-9b4f395d-32e7-4a97-b271-dcd786fcd5d0 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:50.227902 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-9b4f395d-32e7-4a97-b271-dcd786fcd5d0 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: bd0f42d4-7cae-4736-8c36-fe5fa144908e] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:50.234531 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-9b4f395d-32e7-4a97-b271-dcd786fcd5d0 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:50.235002 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-9b4f395d-32e7-4a97-b271-dcd786fcd5d0 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:50.235430 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-9b4f395d-32e7-4a97-b271-dcd786fcd5d0 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:50.236103 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-9b4f395d-32e7-4a97-b271-dcd786fcd5d0 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:50.236551 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-9b4f395d-32e7-4a97-b271-dcd786fcd5d0 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:50.236997 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-9b4f395d-32e7-4a97-b271-dcd786fcd5d0 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:50.364842 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-9b4f395d-32e7-4a97-b271-dcd786fcd5d0 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5e3c891f2f3148f4aaba41a14d507cdf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7203c52ae1f946bd9717a554316e4f4c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:50.372599 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-9b4f395d-32e7-4a97-b271-dcd786fcd5d0 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "GET /compute/v2.1/servers/bd0f42d4-7cae-4736-8c36-fe5fa144908e" status: 200 len: 1514 microversion: 2.1 time: 0.149131 Aug 08 02:30:50.373198 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 359/720] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:50 2022] GET /compute/v2.1/servers/bd0f42d4-7cae-4736-8c36-fe5fa144908e => generated 1514 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:50.677982 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-ad6bb458-4ef9-45f7-9e68-43f79a77b98c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:50.680161 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-ad6bb458-4ef9-45f7-9e68-43f79a77b98c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: aa20d25c-bd00-41e6-b350-309368839578] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:50.687594 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ad6bb458-4ef9-45f7-9e68-43f79a77b98c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:50.688087 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ad6bb458-4ef9-45f7-9e68-43f79a77b98c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:50.688836 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ad6bb458-4ef9-45f7-9e68-43f79a77b98c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:50.689246 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ad6bb458-4ef9-45f7-9e68-43f79a77b98c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:50.689696 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ad6bb458-4ef9-45f7-9e68-43f79a77b98c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:50.690121 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ad6bb458-4ef9-45f7-9e68-43f79a77b98c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:50.720978 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-ad6bb458-4ef9-45f7-9e68-43f79a77b98c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.046858 Aug 08 02:30:50.721555 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 362/721] 10.222.0.27 () {58 vars in 1302 bytes} [Mon Aug 8 02:30:50 2022] GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578/os-volume_attachments => generated 25 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Aug 08 02:30:50.788226 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-dfbf4df0-1fcb-4b5b-ac9c-d6017d8aa9cd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:50.788802 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-dfbf4df0-1fcb-4b5b-ac9c-d6017d8aa9cd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: aa20d25c-bd00-41e6-b350-309368839578] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:50.795632 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-dfbf4df0-1fcb-4b5b-ac9c-d6017d8aa9cd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:50.796112 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-dfbf4df0-1fcb-4b5b-ac9c-d6017d8aa9cd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:50.796545 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-dfbf4df0-1fcb-4b5b-ac9c-d6017d8aa9cd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:50.797257 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-dfbf4df0-1fcb-4b5b-ac9c-d6017d8aa9cd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:50.797705 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-dfbf4df0-1fcb-4b5b-ac9c-d6017d8aa9cd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:50.798120 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-dfbf4df0-1fcb-4b5b-ac9c-d6017d8aa9cd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:50.825660 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-dfbf4df0-1fcb-4b5b-ac9c-d6017d8aa9cd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: aa20d25c-bd00-41e6-b350-309368839578] Going to try to terminate instance {{(pid=118351) delete /opt/stack/nova/nova/compute/api.py:2666}} Aug 08 02:30:50.861392 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-db1e2096-c31d-47e7-837f-b0beb4127e9c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:50.861896 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-db1e2096-c31d-47e7-837f-b0beb4127e9c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: ed75f58f-9135-4d97-b24a-2f8fcdab53ce] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:50.867837 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-db1e2096-c31d-47e7-837f-b0beb4127e9c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:50.868323 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-db1e2096-c31d-47e7-837f-b0beb4127e9c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:50.868787 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-db1e2096-c31d-47e7-837f-b0beb4127e9c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:50.871169 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-db1e2096-c31d-47e7-837f-b0beb4127e9c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:50.871169 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-db1e2096-c31d-47e7-837f-b0beb4127e9c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:50.871169 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-db1e2096-c31d-47e7-837f-b0beb4127e9c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:51.007779 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-db1e2096-c31d-47e7-837f-b0beb4127e9c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:51.016311 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-db1e2096-c31d-47e7-837f-b0beb4127e9c tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce" status: 200 len: 1522 microversion: 2.1 time: 0.161072 Aug 08 02:30:51.016937 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 363/722] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:50 2022] GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce => generated 1522 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:51.019477 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_api:SELECT=97,nova_api:UPDATE=3 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:30:51.098063 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-dfbf4df0-1fcb-4b5b-ac9c-d6017d8aa9cd tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "DELETE /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578" status: 204 len: 0 microversion: 2.1 time: 0.313327 Aug 08 02:30:51.098648 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 360/723] 10.222.0.27 () {58 vars in 1239 bytes} [Mon Aug 8 02:30:50 2022] DELETE /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 08 02:30:51.107737 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-1fb8b05a-d4e3-4ce8-8366-fbbcbc03b6e2 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:51.108357 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-1fb8b05a-d4e3-4ce8-8366-fbbcbc03b6e2 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: aa20d25c-bd00-41e6-b350-309368839578] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:51.114552 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1fb8b05a-d4e3-4ce8-8366-fbbcbc03b6e2 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:51.115003 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1fb8b05a-d4e3-4ce8-8366-fbbcbc03b6e2 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:51.115451 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1fb8b05a-d4e3-4ce8-8366-fbbcbc03b6e2 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:51.116150 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1fb8b05a-d4e3-4ce8-8366-fbbcbc03b6e2 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:51.116611 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1fb8b05a-d4e3-4ce8-8366-fbbcbc03b6e2 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:51.117114 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1fb8b05a-d4e3-4ce8-8366-fbbcbc03b6e2 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:51.266574 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-1fb8b05a-d4e3-4ce8-8366-fbbcbc03b6e2 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:51.274512 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-1fb8b05a-d4e3-4ce8-8366-fbbcbc03b6e2 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578" status: 200 len: 1678 microversion: 2.1 time: 0.171328 Aug 08 02:30:51.275090 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 364/724] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:51 2022] GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578 => generated 1678 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:51.394576 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-728cd161-4170-4994-bbb1-65c98d0e9e0f tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:51.396135 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-728cd161-4170-4994-bbb1-65c98d0e9e0f tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: bd0f42d4-7cae-4736-8c36-fe5fa144908e] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:51.401813 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-728cd161-4170-4994-bbb1-65c98d0e9e0f tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:51.402305 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-728cd161-4170-4994-bbb1-65c98d0e9e0f tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:51.402749 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-728cd161-4170-4994-bbb1-65c98d0e9e0f tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:51.403443 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-728cd161-4170-4994-bbb1-65c98d0e9e0f tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:51.403904 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-728cd161-4170-4994-bbb1-65c98d0e9e0f tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:51.404332 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-728cd161-4170-4994-bbb1-65c98d0e9e0f tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:51.483575 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-728cd161-4170-4994-bbb1-65c98d0e9e0f tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5e3c891f2f3148f4aaba41a14d507cdf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7203c52ae1f946bd9717a554316e4f4c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:51.492396 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-728cd161-4170-4994-bbb1-65c98d0e9e0f tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "GET /compute/v2.1/servers/bd0f42d4-7cae-4736-8c36-fe5fa144908e" status: 200 len: 1472 microversion: 2.1 time: 0.102093 Aug 08 02:30:51.495160 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 361/725] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:51 2022] GET /compute/v2.1/servers/bd0f42d4-7cae-4736-8c36-fe5fa144908e => generated 1472 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:51.592129 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [req-5573758d-44bb-4b4f-ba7f-4b30a525b86f req-ba8cc955-30bf-46e4-86c0-0d8b2c07b129 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bd0f42d4-7cae-4736-8c36-fe5fa144908e", "name": "network-vif-deleted", "tag": "45cf1df5-f1d8-432f-8520-f0edf1de75df"}]} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:30:51.601769 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-5573758d-44bb-4b4f-ba7f-4b30a525b86f req-ba8cc955-30bf-46e4-86c0-0d8b2c07b129 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:51.602269 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-5573758d-44bb-4b4f-ba7f-4b30a525b86f req-ba8cc955-30bf-46e4-86c0-0d8b2c07b129 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:51.602683 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-5573758d-44bb-4b4f-ba7f-4b30a525b86f req-ba8cc955-30bf-46e4-86c0-0d8b2c07b129 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:51.614399 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.compute.server_external_events [req-5573758d-44bb-4b4f-ba7f-4b30a525b86f req-ba8cc955-30bf-46e4-86c0-0d8b2c07b129 service nova] Creating event network-vif-deleted:45cf1df5-f1d8-432f-8520-f0edf1de75df for instance bd0f42d4-7cae-4736-8c36-fe5fa144908e on hv-cinder-74014 Aug 08 02:30:51.619345 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [req-5573758d-44bb-4b4f-ba7f-4b30a525b86f req-ba8cc955-30bf-46e4-86c0-0d8b2c07b129 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031565 Aug 08 02:30:51.619922 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 365/726] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:30:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:30:52.041939 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-91eeae9b-b8b6-4c7e-85ef-e9a9ac4735e0 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:52.042580 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-91eeae9b-b8b6-4c7e-85ef-e9a9ac4735e0 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: ed75f58f-9135-4d97-b24a-2f8fcdab53ce] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:52.048975 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-91eeae9b-b8b6-4c7e-85ef-e9a9ac4735e0 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:52.049452 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-91eeae9b-b8b6-4c7e-85ef-e9a9ac4735e0 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:52.049886 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-91eeae9b-b8b6-4c7e-85ef-e9a9ac4735e0 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:52.050568 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-91eeae9b-b8b6-4c7e-85ef-e9a9ac4735e0 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:52.051017 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-91eeae9b-b8b6-4c7e-85ef-e9a9ac4735e0 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:52.051439 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-91eeae9b-b8b6-4c7e-85ef-e9a9ac4735e0 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:52.117145 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-91eeae9b-b8b6-4c7e-85ef-e9a9ac4735e0 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:52.124780 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-91eeae9b-b8b6-4c7e-85ef-e9a9ac4735e0 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce" status: 200 len: 1480 microversion: 2.1 time: 0.086819 Aug 08 02:30:52.125339 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 362/727] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:52 2022] GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce => generated 1480 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:52.296804 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-67252637-c78d-434b-8eb2-f21ec0fc5dbe tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:52.297486 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-67252637-c78d-434b-8eb2-f21ec0fc5dbe tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: aa20d25c-bd00-41e6-b350-309368839578] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:52.303622 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-67252637-c78d-434b-8eb2-f21ec0fc5dbe tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:52.304095 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-67252637-c78d-434b-8eb2-f21ec0fc5dbe tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:52.304525 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-67252637-c78d-434b-8eb2-f21ec0fc5dbe tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:52.305229 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-67252637-c78d-434b-8eb2-f21ec0fc5dbe tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:52.305676 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-67252637-c78d-434b-8eb2-f21ec0fc5dbe tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:52.306098 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-67252637-c78d-434b-8eb2-f21ec0fc5dbe tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:52.438110 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-67252637-c78d-434b-8eb2-f21ec0fc5dbe tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:52.446060 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-67252637-c78d-434b-8eb2-f21ec0fc5dbe tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578" status: 200 len: 1678 microversion: 2.1 time: 0.153866 Aug 08 02:30:52.446645 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 366/728] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:52 2022] GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578 => generated 1678 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:52.514484 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-ae26cf4b-0c6f-479f-9504-92ca68d132f6 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:52.515085 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-ae26cf4b-0c6f-479f-9504-92ca68d132f6 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: bd0f42d4-7cae-4736-8c36-fe5fa144908e] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:52.521109 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ae26cf4b-0c6f-479f-9504-92ca68d132f6 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:52.521588 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ae26cf4b-0c6f-479f-9504-92ca68d132f6 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:52.522022 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ae26cf4b-0c6f-479f-9504-92ca68d132f6 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:52.522703 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ae26cf4b-0c6f-479f-9504-92ca68d132f6 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:52.523193 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ae26cf4b-0c6f-479f-9504-92ca68d132f6 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:52.523612 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ae26cf4b-0c6f-479f-9504-92ca68d132f6 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:52.587674 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-ae26cf4b-0c6f-479f-9504-92ca68d132f6 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5e3c891f2f3148f4aaba41a14d507cdf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7203c52ae1f946bd9717a554316e4f4c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:52.596625 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-ae26cf4b-0c6f-479f-9504-92ca68d132f6 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "GET /compute/v2.1/servers/bd0f42d4-7cae-4736-8c36-fe5fa144908e" status: 200 len: 1472 microversion: 2.1 time: 0.086093 Aug 08 02:30:52.597215 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 363/729] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:52 2022] GET /compute/v2.1/servers/bd0f42d4-7cae-4736-8c36-fe5fa144908e => generated 1472 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:53.145955 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-e49241bc-e393-4b65-a3de-f90d9e95f915 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:53.146565 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-e49241bc-e393-4b65-a3de-f90d9e95f915 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: ed75f58f-9135-4d97-b24a-2f8fcdab53ce] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:53.152591 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e49241bc-e393-4b65-a3de-f90d9e95f915 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:53.153115 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e49241bc-e393-4b65-a3de-f90d9e95f915 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:53.153760 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e49241bc-e393-4b65-a3de-f90d9e95f915 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:53.154466 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e49241bc-e393-4b65-a3de-f90d9e95f915 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:53.154933 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e49241bc-e393-4b65-a3de-f90d9e95f915 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:53.155361 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e49241bc-e393-4b65-a3de-f90d9e95f915 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:53.186174 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e49241bc-e393-4b65-a3de-f90d9e95f915 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:53.186812 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e49241bc-e393-4b65-a3de-f90d9e95f915 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:53.187273 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e49241bc-e393-4b65-a3de-f90d9e95f915 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:53.187743 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.objects.instance [None req-e49241bc-e393-4b65-a3de-f90d9e95f915 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lazy-loading 'fault' on Instance uuid ed75f58f-9135-4d97-b24a-2f8fcdab53ce {{(pid=118352) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 08 02:30:53.231302 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-e49241bc-e393-4b65-a3de-f90d9e95f915 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5c8ae69712c14878bd2aab6d33512b7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8af7d2bbc6d94273bd39c81423d8cebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:53.239047 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-e49241bc-e393-4b65-a3de-f90d9e95f915 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce" status: 200 len: 1312 microversion: 2.1 time: 0.097194 Aug 08 02:30:53.239753 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 367/730] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:53 2022] GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce => generated 1312 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:53.468513 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-40d9efe1-9349-41d5-ace9-6961e61d9670 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:53.472149 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-40d9efe1-9349-41d5-ace9-6961e61d9670 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: aa20d25c-bd00-41e6-b350-309368839578] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:53.479431 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-40d9efe1-9349-41d5-ace9-6961e61d9670 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:53.481728 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-40d9efe1-9349-41d5-ace9-6961e61d9670 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:53.481728 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-40d9efe1-9349-41d5-ace9-6961e61d9670 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:53.481728 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-40d9efe1-9349-41d5-ace9-6961e61d9670 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:53.482195 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-40d9efe1-9349-41d5-ace9-6961e61d9670 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:53.482374 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-40d9efe1-9349-41d5-ace9-6961e61d9670 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:53.553168 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-40d9efe1-9349-41d5-ace9-6961e61d9670 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:53.561734 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-40d9efe1-9349-41d5-ace9-6961e61d9670 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578" status: 200 len: 1612 microversion: 2.1 time: 0.097269 Aug 08 02:30:53.562309 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 364/731] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:53 2022] GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578 => generated 1612 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:30:53.618032 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-dce0e68a-05a6-4cdb-a044-9c69216b5789 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:53.618755 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-dce0e68a-05a6-4cdb-a044-9c69216b5789 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: bd0f42d4-7cae-4736-8c36-fe5fa144908e] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:53.629718 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dce0e68a-05a6-4cdb-a044-9c69216b5789 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:53.629718 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dce0e68a-05a6-4cdb-a044-9c69216b5789 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:53.629718 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dce0e68a-05a6-4cdb-a044-9c69216b5789 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:53.629718 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dce0e68a-05a6-4cdb-a044-9c69216b5789 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:53.630076 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dce0e68a-05a6-4cdb-a044-9c69216b5789 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:53.630632 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dce0e68a-05a6-4cdb-a044-9c69216b5789 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:53.651332 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:30:53.665628 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dce0e68a-05a6-4cdb-a044-9c69216b5789 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:53.666877 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dce0e68a-05a6-4cdb-a044-9c69216b5789 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:53.667636 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dce0e68a-05a6-4cdb-a044-9c69216b5789 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:53.668407 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.objects.instance [None req-dce0e68a-05a6-4cdb-a044-9c69216b5789 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lazy-loading 'fault' on Instance uuid bd0f42d4-7cae-4736-8c36-fe5fa144908e {{(pid=118352) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 08 02:30:53.717371 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-dce0e68a-05a6-4cdb-a044-9c69216b5789 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5e3c891f2f3148f4aaba41a14d507cdf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7203c52ae1f946bd9717a554316e4f4c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:53.730254 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [req-b2604110-3ce6-48da-9bc8-2838c0018276 req-9eb29a83-c699-455a-95c6-3a95493cdb87 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ed75f58f-9135-4d97-b24a-2f8fcdab53ce", "name": "network-vif-deleted", "tag": "ee2c6536-d687-4d7c-aac6-d3b85f022762"}]} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:30:53.744112 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-dce0e68a-05a6-4cdb-a044-9c69216b5789 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "GET /compute/v2.1/servers/bd0f42d4-7cae-4736-8c36-fe5fa144908e" status: 200 len: 1309 microversion: 2.1 time: 0.130004 Aug 08 02:30:53.747974 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 368/732] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:53 2022] GET /compute/v2.1/servers/bd0f42d4-7cae-4736-8c36-fe5fa144908e => generated 1309 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:30:53.755018 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-b2604110-3ce6-48da-9bc8-2838c0018276 req-9eb29a83-c699-455a-95c6-3a95493cdb87 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:53.758824 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-b2604110-3ce6-48da-9bc8-2838c0018276 req-9eb29a83-c699-455a-95c6-3a95493cdb87 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:53.759433 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-b2604110-3ce6-48da-9bc8-2838c0018276 req-9eb29a83-c699-455a-95c6-3a95493cdb87 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:53.784081 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.compute.server_external_events [req-b2604110-3ce6-48da-9bc8-2838c0018276 req-9eb29a83-c699-455a-95c6-3a95493cdb87 service nova] Creating event network-vif-deleted:ee2c6536-d687-4d7c-aac6-d3b85f022762 for instance ed75f58f-9135-4d97-b24a-2f8fcdab53ce on hv-cinder-74014 Aug 08 02:30:53.794556 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [req-b2604110-3ce6-48da-9bc8-2838c0018276 req-9eb29a83-c699-455a-95c6-3a95493cdb87 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.162734 Aug 08 02:30:53.795181 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 365/733] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:30:53 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:30:54.260913 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-ab21a333-9516-4f8b-b111-38f41165ad66 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:54.261610 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-ab21a333-9516-4f8b-b111-38f41165ad66 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] [instance: ed75f58f-9135-4d97-b24a-2f8fcdab53ce] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:54.267826 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ab21a333-9516-4f8b-b111-38f41165ad66 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:54.268312 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ab21a333-9516-4f8b-b111-38f41165ad66 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:54.268783 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ab21a333-9516-4f8b-b111-38f41165ad66 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:54.269479 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ab21a333-9516-4f8b-b111-38f41165ad66 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:54.269936 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ab21a333-9516-4f8b-b111-38f41165ad66 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:54.270365 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ab21a333-9516-4f8b-b111-38f41165ad66 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:54.284050 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.wsgi [None req-ab21a333-9516-4f8b-b111-38f41165ad66 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] HTTP exception thrown: Instance ed75f58f-9135-4d97-b24a-2f8fcdab53ce could not be found. Aug 08 02:30:54.285739 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-ab21a333-9516-4f8b-b111-38f41165ad66 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Request method failure captured: Aug 08 02:30:54.285739 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: request: GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce HTTP/1.1 Aug 08 02:30:54.285739 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept: application/json Aug 08 02:30:54.285739 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept-Encoding: identity Aug 08 02:30:54.285739 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Connection: close Aug 08 02:30:54.285739 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Length: 0 Aug 08 02:30:54.285739 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Type: application/json Aug 08 02:30:54.285739 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Host: 10.222.0.27 Aug 08 02:30:54.285739 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Openstack-System-Scope: None Aug 08 02:30:54.285739 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: User-Agent: python-urllib3/1.26.11 Aug 08 02:30:54.285739 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Auth-Token: gAAAAABi8HQXsGK5SLdeRHabROI8DgUVfT8-qt3OmjigKWIwr2r9ylfDom5hOf3PBwXXbVpQKaQQCg2GWwYCANhQNXkemxQtH2Z8vABNoiUbOWpRK-0iPQcLUey3hJs3m48mZ7E_Wt-sxx-vNwT0dxN6KL1UanLlX0GY-h9OUcxTwCxsePW3BgQ Aug 08 02:30:54.285739 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Id: None Aug 08 02:30:54.285739 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Name: None Aug 08 02:30:54.285739 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Identity-Status: Confirmed Aug 08 02:30:54.285739 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Is-Admin-Project: True Aug 08 02:30:54.285739 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Id: default Aug 08 02:30:54.285739 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Name: Default Aug 08 02:30:54.285739 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Id: 8af7d2bbc6d94273bd39c81423d8cebe Aug 08 02:30:54.285739 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-514896148 Aug 08 02:30:54.285739 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Role: reader,member Aug 08 02:30:54.285739 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Roles: reader,member Aug 08 02:30:54.287157 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/8af7d2bbc6d94273bd39c81423d8cebe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/8af7d2bbc6d94273bd39c81423d8cebe"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_8af7d2bbc6d94273bd39c81423d8cebe", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/8af7d2bbc6d94273bd39c81423d8cebe"}]}] Aug 08 02:30:54.287157 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant: tempest-ServerRescueNegativeTestJSON-514896148 Aug 08 02:30:54.287157 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Id: 8af7d2bbc6d94273bd39c81423d8cebe Aug 08 02:30:54.287157 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-514896148 Aug 08 02:30:54.287157 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User: tempest-ServerRescueNegativeTestJSON-514896148-project Aug 08 02:30:54.287157 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Id: default Aug 08 02:30:54.287157 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Name: Default Aug 08 02:30:54.287157 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Id: 5c8ae69712c14878bd2aab6d33512b7a Aug 08 02:30:54.287157 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Name: tempest-ServerRescueNegativeTestJSON-514896148-project Aug 08 02:30:54.287157 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: method: > Aug 08 02:30:54.287157 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: exception: Instance ed75f58f-9135-4d97-b24a-2f8fcdab53ce could not be found. Aug 08 02:30:54.287157 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: action_args: {'id': 'ed75f58f-9135-4d97-b24a-2f8fcdab53ce'} Aug 08 02:30:54.287922 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:30:54.287922 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:30:54.287922 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 08 02:30:54.287922 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 08 02:30:54.287922 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 08 02:30:54.287922 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 08 02:30:54.287922 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 08 02:30:54.287922 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 08 02:30:54.287922 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 08 02:30:54.287922 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 08 02:30:54.287922 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ed75f58f-9135-4d97-b24a-2f8fcdab53ce could not be found. Aug 08 02:30:54.287922 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:30:54.287922 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:30:54.287922 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:30:54.287922 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:30:54.287922 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:30:54.287922 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:30:54.287922 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:30:54.287922 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:30:54.287922 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:30:54.289145 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:30:54.289145 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 08 02:30:54.289145 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 08 02:30:54.289145 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 08 02:30:54.289145 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 08 02:30:54.289145 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 08 02:30:54.289145 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:30:54.289145 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ed75f58f-9135-4d97-b24a-2f8fcdab53ce could not be found. Aug 08 02:30:54.289145 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:30:54.289145 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:30:54.289145 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:30:54.289145 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:30:54.289145 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:30:54.289145 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:30:54.289145 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:30:54.289145 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:30:54.289145 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ed75f58f-9135-4d97-b24a-2f8fcdab53ce could not be found. Aug 08 02:30:54.289145 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:30:54.290185 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-ab21a333-9516-4f8b-b111-38f41165ad66 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] Returning 404 to user: Instance ed75f58f-9135-4d97-b24a-2f8fcdab53ce could not be found. {{(pid=118352) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:30:54.290185 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-ab21a333-9516-4f8b-b111-38f41165ad66 tempest-ServerRescueNegativeTestJSON-514896148 tempest-ServerRescueNegativeTestJSON-514896148-project] 10.222.0.27 "GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce" status: 404 len: 111 microversion: 2.1 time: 0.030596 Aug 08 02:30:54.290185 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 369/734] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:54 2022] GET /compute/v2.1/servers/ed75f58f-9135-4d97-b24a-2f8fcdab53ce => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 08 02:30:54.584867 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-78570317-5f1e-4a24-9f0e-de7630454357 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:54.586168 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-78570317-5f1e-4a24-9f0e-de7630454357 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: aa20d25c-bd00-41e6-b350-309368839578] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:54.602017 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-78570317-5f1e-4a24-9f0e-de7630454357 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:54.602626 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-78570317-5f1e-4a24-9f0e-de7630454357 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:54.603140 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-78570317-5f1e-4a24-9f0e-de7630454357 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:54.605594 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-78570317-5f1e-4a24-9f0e-de7630454357 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:54.606153 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-78570317-5f1e-4a24-9f0e-de7630454357 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:54.606658 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-78570317-5f1e-4a24-9f0e-de7630454357 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:54.628621 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell1:SELECT=95,nova_cell1:INSERT=3,nova_cell1:UPDATE=2 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:30:54.644625 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-78570317-5f1e-4a24-9f0e-de7630454357 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:54.645210 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-78570317-5f1e-4a24-9f0e-de7630454357 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:54.645642 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-78570317-5f1e-4a24-9f0e-de7630454357 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:54.646146 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.objects.instance [None req-78570317-5f1e-4a24-9f0e-de7630454357 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lazy-loading 'fault' on Instance uuid aa20d25c-bd00-41e6-b350-309368839578 {{(pid=118351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 08 02:30:54.719088 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-78570317-5f1e-4a24-9f0e-de7630454357 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:30:54.730790 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-78570317-5f1e-4a24-9f0e-de7630454357 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578" status: 200 len: 1331 microversion: 2.1 time: 0.149882 Aug 08 02:30:54.732359 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 366/735] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:54 2022] GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578 => generated 1331 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:30:54.766949 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-ee97f51d-9e39-4816-8f72-36272d520187 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:54.768169 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-ee97f51d-9e39-4816-8f72-36272d520187 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] [instance: bd0f42d4-7cae-4736-8c36-fe5fa144908e] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:54.774715 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ee97f51d-9e39-4816-8f72-36272d520187 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:54.782870 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ee97f51d-9e39-4816-8f72-36272d520187 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:54.782870 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ee97f51d-9e39-4816-8f72-36272d520187 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:54.782870 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ee97f51d-9e39-4816-8f72-36272d520187 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:54.782870 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ee97f51d-9e39-4816-8f72-36272d520187 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:54.783255 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ee97f51d-9e39-4816-8f72-36272d520187 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:54.793588 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.wsgi [None req-ee97f51d-9e39-4816-8f72-36272d520187 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] HTTP exception thrown: Instance bd0f42d4-7cae-4736-8c36-fe5fa144908e could not be found. Aug 08 02:30:54.795490 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-ee97f51d-9e39-4816-8f72-36272d520187 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Request method failure captured: Aug 08 02:30:54.795490 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: request: GET /compute/v2.1/servers/bd0f42d4-7cae-4736-8c36-fe5fa144908e HTTP/1.1 Aug 08 02:30:54.795490 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept: application/json Aug 08 02:30:54.795490 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept-Encoding: identity Aug 08 02:30:54.795490 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Connection: close Aug 08 02:30:54.795490 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Length: 0 Aug 08 02:30:54.795490 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Type: application/json Aug 08 02:30:54.795490 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Host: 10.222.0.27 Aug 08 02:30:54.795490 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Openstack-System-Scope: None Aug 08 02:30:54.795490 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: User-Agent: python-urllib3/1.26.11 Aug 08 02:30:54.795490 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Auth-Token: gAAAAABi8HTH8a0CBQ8lEnnKCKkC_gQfnY8t7Ey_gArOYoTtLvqT8lOtRp95WoRUY5Q1WdPf_EGbzi8cT3to_wING_-blz9aiXVpDzS907isS90uSr-YeThzVTF_ORbCFIluGwDnAYt15m-PmK6CNHU-RauOpu9WoCM938azTnTeZfmzXoi2yzo Aug 08 02:30:54.795490 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Id: None Aug 08 02:30:54.795490 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Name: None Aug 08 02:30:54.795490 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Identity-Status: Confirmed Aug 08 02:30:54.795490 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Is-Admin-Project: True Aug 08 02:30:54.795490 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Id: default Aug 08 02:30:54.795490 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Name: Default Aug 08 02:30:54.795490 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Id: 7203c52ae1f946bd9717a554316e4f4c Aug 08 02:30:54.795490 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Name: tempest-VolumesSnapshotTestJSON-71153930 Aug 08 02:30:54.795490 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Role: reader,member Aug 08 02:30:54.795490 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Roles: reader,member Aug 08 02:30:54.796778 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/7203c52ae1f946bd9717a554316e4f4c"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/7203c52ae1f946bd9717a554316e4f4c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_7203c52ae1f946bd9717a554316e4f4c", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/7203c52ae1f946bd9717a554316e4f4c"}]}] Aug 08 02:30:54.796778 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant: tempest-VolumesSnapshotTestJSON-71153930 Aug 08 02:30:54.796778 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Id: 7203c52ae1f946bd9717a554316e4f4c Aug 08 02:30:54.796778 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-71153930 Aug 08 02:30:54.796778 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User: tempest-VolumesSnapshotTestJSON-71153930-project Aug 08 02:30:54.796778 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Id: default Aug 08 02:30:54.796778 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Name: Default Aug 08 02:30:54.796778 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Id: 5e3c891f2f3148f4aaba41a14d507cdf Aug 08 02:30:54.796778 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Name: tempest-VolumesSnapshotTestJSON-71153930-project Aug 08 02:30:54.796778 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: method: > Aug 08 02:30:54.796778 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: exception: Instance bd0f42d4-7cae-4736-8c36-fe5fa144908e could not be found. Aug 08 02:30:54.796778 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: action_args: {'id': 'bd0f42d4-7cae-4736-8c36-fe5fa144908e'} Aug 08 02:30:54.797534 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:30:54.797534 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:30:54.797534 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 08 02:30:54.797534 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 08 02:30:54.797534 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 08 02:30:54.797534 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 08 02:30:54.797534 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 08 02:30:54.797534 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 08 02:30:54.797534 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 08 02:30:54.797534 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 08 02:30:54.797534 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance bd0f42d4-7cae-4736-8c36-fe5fa144908e could not be found. Aug 08 02:30:54.797534 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:30:54.797534 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:30:54.797534 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:30:54.797534 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:30:54.797534 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:30:54.797534 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:30:54.797534 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:30:54.797534 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:30:54.797534 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:30:54.798681 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:30:54.798681 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 08 02:30:54.798681 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 08 02:30:54.798681 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 08 02:30:54.798681 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 08 02:30:54.798681 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 08 02:30:54.798681 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:30:54.798681 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance bd0f42d4-7cae-4736-8c36-fe5fa144908e could not be found. Aug 08 02:30:54.798681 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:30:54.798681 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:30:54.798681 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:30:54.798681 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:30:54.798681 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:30:54.798681 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:30:54.798681 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:30:54.798681 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:30:54.798681 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance bd0f42d4-7cae-4736-8c36-fe5fa144908e could not be found. Aug 08 02:30:54.798681 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:30:54.800825 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-ee97f51d-9e39-4816-8f72-36272d520187 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] Returning 404 to user: Instance bd0f42d4-7cae-4736-8c36-fe5fa144908e could not be found. {{(pid=118352) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:30:54.800825 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-ee97f51d-9e39-4816-8f72-36272d520187 tempest-VolumesSnapshotTestJSON-71153930 tempest-VolumesSnapshotTestJSON-71153930-project] 10.222.0.27 "GET /compute/v2.1/servers/bd0f42d4-7cae-4736-8c36-fe5fa144908e" status: 404 len: 111 microversion: 2.1 time: 0.035147 Aug 08 02:30:54.800825 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 370/736] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:54 2022] GET /compute/v2.1/servers/bd0f42d4-7cae-4736-8c36-fe5fa144908e => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 08 02:30:55.757047 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-a7a99122-95d0-4e83-a960-ae93c30b0262 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:55.757773 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-a7a99122-95d0-4e83-a960-ae93c30b0262 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: aa20d25c-bd00-41e6-b350-309368839578] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:30:55.765682 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a7a99122-95d0-4e83-a960-ae93c30b0262 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:55.766169 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a7a99122-95d0-4e83-a960-ae93c30b0262 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:55.766931 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a7a99122-95d0-4e83-a960-ae93c30b0262 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:55.767707 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a7a99122-95d0-4e83-a960-ae93c30b0262 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:55.768360 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a7a99122-95d0-4e83-a960-ae93c30b0262 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:55.768881 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a7a99122-95d0-4e83-a960-ae93c30b0262 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:55.784366 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.wsgi [None req-a7a99122-95d0-4e83-a960-ae93c30b0262 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] HTTP exception thrown: Instance aa20d25c-bd00-41e6-b350-309368839578 could not be found. Aug 08 02:30:55.786158 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-a7a99122-95d0-4e83-a960-ae93c30b0262 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Request method failure captured: Aug 08 02:30:55.786158 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: request: GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578 HTTP/1.1 Aug 08 02:30:55.786158 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Accept: application/json Aug 08 02:30:55.786158 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Accept-Encoding: identity Aug 08 02:30:55.786158 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Connection: close Aug 08 02:30:55.786158 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Content-Length: 0 Aug 08 02:30:55.786158 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Content-Type: application/json Aug 08 02:30:55.786158 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Host: 10.222.0.27 Aug 08 02:30:55.786158 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Openstack-System-Scope: None Aug 08 02:30:55.786158 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: User-Agent: python-urllib3/1.26.11 Aug 08 02:30:55.786158 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Auth-Token: gAAAAABi8HQUs7QI_wjv0vSDVlijMnZ9-Lkgqc--Bu80jwWL-CoTBjbpe1sae_INrRIGeFqgwK-ezXnEOgqpG2NO5P97SEgPcY4zY_QgnaTA6xv-rzef5kDF5s4pU5M8QdM-T8eUF020_v3zsmgR78rxXaglg0C5QuxYvT8R4THTJZuwa_f26Bo Aug 08 02:30:55.786158 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Domain-Id: None Aug 08 02:30:55.786158 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Domain-Name: None Aug 08 02:30:55.786158 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Identity-Status: Confirmed Aug 08 02:30:55.786158 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Is-Admin-Project: True Aug 08 02:30:55.786158 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Domain-Id: default Aug 08 02:30:55.786158 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Domain-Name: Default Aug 08 02:30:55.786158 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Id: ca29b2b9a59d4826bfd25f4dcfa9cffe Aug 08 02:30:55.786158 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Name: tempest-AttachVolumeNegativeTest-420450256 Aug 08 02:30:55.786158 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Role: reader,member Aug 08 02:30:55.786158 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Roles: reader,member Aug 08 02:30:55.787808 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/ca29b2b9a59d4826bfd25f4dcfa9cffe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/ca29b2b9a59d4826bfd25f4dcfa9cffe"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_ca29b2b9a59d4826bfd25f4dcfa9cffe", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/ca29b2b9a59d4826bfd25f4dcfa9cffe"}]}] Aug 08 02:30:55.787808 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant: tempest-AttachVolumeNegativeTest-420450256 Aug 08 02:30:55.787808 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant-Id: ca29b2b9a59d4826bfd25f4dcfa9cffe Aug 08 02:30:55.787808 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-420450256 Aug 08 02:30:55.787808 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User: tempest-AttachVolumeNegativeTest-420450256-project Aug 08 02:30:55.787808 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Domain-Id: default Aug 08 02:30:55.787808 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Domain-Name: Default Aug 08 02:30:55.787808 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Id: 578cd837d9fe442587f8c6c8528334c0 Aug 08 02:30:55.787808 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Name: tempest-AttachVolumeNegativeTest-420450256-project Aug 08 02:30:55.787808 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: method: > Aug 08 02:30:55.787808 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: exception: Instance aa20d25c-bd00-41e6-b350-309368839578 could not be found. Aug 08 02:30:55.787808 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: action_args: {'id': 'aa20d25c-bd00-41e6-b350-309368839578'} Aug 08 02:30:55.788878 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:30:55.788878 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:30:55.788878 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 08 02:30:55.788878 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 08 02:30:55.788878 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 08 02:30:55.788878 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 08 02:30:55.788878 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 08 02:30:55.788878 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 08 02:30:55.788878 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 08 02:30:55.788878 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 08 02:30:55.788878 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance aa20d25c-bd00-41e6-b350-309368839578 could not be found. Aug 08 02:30:55.788878 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:30:55.788878 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:30:55.788878 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:30:55.788878 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:30:55.788878 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:30:55.788878 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:30:55.788878 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:30:55.788878 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:30:55.788878 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:30:55.791240 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:30:55.791240 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 08 02:30:55.791240 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 08 02:30:55.791240 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 08 02:30:55.791240 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 08 02:30:55.791240 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 08 02:30:55.791240 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:30:55.791240 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance aa20d25c-bd00-41e6-b350-309368839578 could not be found. Aug 08 02:30:55.791240 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:30:55.791240 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:30:55.791240 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:30:55.791240 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:30:55.791240 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:30:55.791240 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:30:55.791240 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:30:55.791240 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:30:55.791240 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance aa20d25c-bd00-41e6-b350-309368839578 could not be found. Aug 08 02:30:55.791240 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:30:55.793636 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-a7a99122-95d0-4e83-a960-ae93c30b0262 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Returning 404 to user: Instance aa20d25c-bd00-41e6-b350-309368839578 could not be found. {{(pid=118351) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:30:55.793636 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-a7a99122-95d0-4e83-a960-ae93c30b0262 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578" status: 404 len: 111 microversion: 2.1 time: 0.038568 Aug 08 02:30:55.793636 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 367/737] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:30:55 2022] GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578 => generated 111 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 08 02:30:55.802135 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-a7a66d72-909b-43b4-b2b7-4b31ef72896d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '.version_select at 0x7efc74060ca0>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:30:55.811389 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [req-57244809-f96b-450d-8aeb-692686d4a4b0 req-c7d24d4a-788f-4a90-a5a2-353d6ce98a93 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "aa20d25c-bd00-41e6-b350-309368839578", "name": "network-vif-deleted", "tag": "72aebc3e-9c8c-4a19-8c44-a4ea8d03d79e"}]} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:30:55.822962 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-57244809-f96b-450d-8aeb-692686d4a4b0 req-c7d24d4a-788f-4a90-a5a2-353d6ce98a93 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:30:55.823673 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-57244809-f96b-450d-8aeb-692686d4a4b0 req-c7d24d4a-788f-4a90-a5a2-353d6ce98a93 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:30:55.824287 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-57244809-f96b-450d-8aeb-692686d4a4b0 req-c7d24d4a-788f-4a90-a5a2-353d6ce98a93 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:30:55.839432 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.compute.server_external_events [req-57244809-f96b-450d-8aeb-692686d4a4b0 req-c7d24d4a-788f-4a90-a5a2-353d6ce98a93 service nova] Creating event network-vif-deleted:72aebc3e-9c8c-4a19-8c44-a4ea8d03d79e for instance aa20d25c-bd00-41e6-b350-309368839578 on hv-cinder-74014 Aug 08 02:30:55.845103 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [req-57244809-f96b-450d-8aeb-692686d4a4b0 req-c7d24d4a-788f-4a90-a5a2-353d6ce98a93 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038697 Aug 08 02:30:55.846313 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 368/738] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:30:55 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 08 02:30:55.901339 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-a7a66d72-909b-43b4-b2b7-4b31ef72896d tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1247410672" status: 202 len: 0 microversion: 2.1 time: 0.103144 Aug 08 02:30:55.902569 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 371/739] 10.222.0.27 () {58 vars in 1221 bytes} [Mon Aug 8 02:30:55 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1247410672 => generated 0 bytes in 106 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 08 02:30:56.813389 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-2f589b9a-c1a2-4f95-b417-ac04e1619b1e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Action: 'create', calling method: .version_select at 0x7efc74096f70>, body: {"keypair": {"name": "tempest-keypair-1786670406", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCoJ4pRNe03e5+Q4WK5I73fi+ElJAzpcOnf9yAuNYpRKfsRuBrjW3/W5Z809Dki976zNt95VnrtTZObzT6hTUcpVpg1T1bKlLiy93g2lc9rvEz/oWoFXRGnYyGhjIiYeVQ=="}} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:30:56.830916 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.quota [None req-2f589b9a-c1a2-4f95-b417-ac04e1619b1e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Getting quotas for project ca29b2b9a59d4826bfd25f4dcfa9cffe. Resources: {'key_pairs'} {{(pid=118351) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 08 02:30:56.835164 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.quota [None req-2f589b9a-c1a2-4f95-b417-ac04e1619b1e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Getting quotas for user 578cd837d9fe442587f8c6c8528334c0 and project ca29b2b9a59d4826bfd25f4dcfa9cffe. Resources: {'key_pairs'} {{(pid=118351) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 08 02:30:56.923250 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-2f589b9a-c1a2-4f95-b417-ac04e1619b1e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.114055 Aug 08 02:30:56.932226 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 369/740] 10.222.0.27 () {60 vars in 1159 bytes} [Mon Aug 8 02:30:56 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:31:00.013438 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-3d561b2e-d21d-4435-9d63-c32a8dd2e4f3 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-392324718", "imageRef": "30833f47-5987-41ca-b5ee-ad5ecafbb265", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "f92c936a-6950-4530-8001-5260ea91adac"}], "security_groups": [{"name": "tempest-securitygroup--936105947"}], "key_name": "tempest-keypair-1786670406", "user_data": "}} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:31:00.126989 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.network.neutron [None req-3d561b2e-d21d-4435-9d63-c32a8dd2e4f3 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] validate_networks() for [('f92c936a-6950-4530-8001-5260ea91adac', None, None, None, None, None)] {{(pid=118352) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 08 02:31:00.612845 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-3d561b2e-d21d-4435-9d63-c32a8dd2e4f3 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Flavor limits 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 08 02:31:00.613299 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-3d561b2e-d21d-4435-9d63-c32a8dd2e4f3 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Image limits 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 08 02:31:00.613846 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-3d561b2e-d21d-4435-9d63-c32a8dd2e4f3 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Flavor pref 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 08 02:31:00.614288 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-3d561b2e-d21d-4435-9d63-c32a8dd2e4f3 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Image pref 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 08 02:31:00.614768 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-3d561b2e-d21d-4435-9d63-c32a8dd2e4f3 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 08 02:31:00.624887 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3d561b2e-d21d-4435-9d63-c32a8dd2e4f3 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:00.625392 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3d561b2e-d21d-4435-9d63-c32a8dd2e4f3 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:00.625845 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3d561b2e-d21d-4435-9d63-c32a8dd2e4f3 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:00.626563 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3d561b2e-d21d-4435-9d63-c32a8dd2e4f3 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:00.627014 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3d561b2e-d21d-4435-9d63-c32a8dd2e4f3 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:00.627434 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3d561b2e-d21d-4435-9d63-c32a8dd2e4f3 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:00.645663 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.quota [None req-3d561b2e-d21d-4435-9d63-c32a8dd2e4f3 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Getting quotas for project ca29b2b9a59d4826bfd25f4dcfa9cffe. Resources: {'ram', 'cores', 'instances'} {{(pid=118352) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 08 02:31:00.649707 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.quota [None req-3d561b2e-d21d-4435-9d63-c32a8dd2e4f3 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Getting quotas for user 578cd837d9fe442587f8c6c8528334c0 and project ca29b2b9a59d4826bfd25f4dcfa9cffe. Resources: {'ram', 'cores', 'instances'} {{(pid=118352) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 08 02:31:00.661930 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-3d561b2e-d21d-4435-9d63-c32a8dd2e4f3 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Going to run 1 instances... {{(pid=118352) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Aug 08 02:31:00.664326 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-3d561b2e-d21d-4435-9d63-c32a8dd2e4f3 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: cc2b42f1-3dd0-4090-a905-93283047aea3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='30833f47-5987-41ca-b5ee-ad5ecafbb265',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118352) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Aug 08 02:31:00.861905 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-3d561b2e-d21d-4435-9d63-c32a8dd2e4f3 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.853595 Aug 08 02:31:00.862555 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 372/741] 10.222.0.27 () {60 vars in 1147 bytes} [Mon Aug 8 02:31:00 2022] POST /compute/v2.1/servers => generated 401 bytes in 855 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 08 02:31:00.874812 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-93c944db-d1c7-4301-9019-cd0a9bd451f2 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:00.875548 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-93c944db-d1c7-4301-9019-cd0a9bd451f2 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: cc2b42f1-3dd0-4090-a905-93283047aea3] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:00.932424 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-93c944db-d1c7-4301-9019-cd0a9bd451f2 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:00.939541 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-93c944db-d1c7-4301-9019-cd0a9bd451f2 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3" status: 200 len: 1241 microversion: 2.1 time: 0.069387 Aug 08 02:31:00.940108 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 370/742] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:31:00 2022] GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3 => generated 1241 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:31:01.961617 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-d218d5a2-3429-4c69-95f8-054bb2d0d156 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:01.964539 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-d218d5a2-3429-4c69-95f8-054bb2d0d156 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: cc2b42f1-3dd0-4090-a905-93283047aea3] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:02.029639 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-d218d5a2-3429-4c69-95f8-054bb2d0d156 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:02.036361 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-d218d5a2-3429-4c69-95f8-054bb2d0d156 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3" status: 200 len: 1241 microversion: 2.1 time: 0.079552 Aug 08 02:31:02.036966 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 373/743] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:31:01 2022] GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3 => generated 1241 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:31:03.056890 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-79b1a2de-caa6-4b10-ac8a-425e9d43fc3a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:03.057455 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-79b1a2de-caa6-4b10-ac8a-425e9d43fc3a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: cc2b42f1-3dd0-4090-a905-93283047aea3] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:03.065110 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-79b1a2de-caa6-4b10-ac8a-425e9d43fc3a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:03.076246 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-79b1a2de-caa6-4b10-ac8a-425e9d43fc3a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:03.076246 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-79b1a2de-caa6-4b10-ac8a-425e9d43fc3a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:03.076246 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-79b1a2de-caa6-4b10-ac8a-425e9d43fc3a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:03.076246 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-79b1a2de-caa6-4b10-ac8a-425e9d43fc3a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:03.076984 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-79b1a2de-caa6-4b10-ac8a-425e9d43fc3a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:03.130032 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-79b1a2de-caa6-4b10-ac8a-425e9d43fc3a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:03.138264 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-79b1a2de-caa6-4b10-ac8a-425e9d43fc3a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3" status: 200 len: 1245 microversion: 2.1 time: 0.085493 Aug 08 02:31:03.138820 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 371/744] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:31:03 2022] GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3 => generated 1245 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:31:04.159950 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-0573bd0e-c1cd-4b9f-8395-194fdccd3524 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:04.161306 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-0573bd0e-c1cd-4b9f-8395-194fdccd3524 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: cc2b42f1-3dd0-4090-a905-93283047aea3] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:04.169850 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0573bd0e-c1cd-4b9f-8395-194fdccd3524 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:04.171149 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0573bd0e-c1cd-4b9f-8395-194fdccd3524 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:04.171805 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0573bd0e-c1cd-4b9f-8395-194fdccd3524 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:04.172806 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0573bd0e-c1cd-4b9f-8395-194fdccd3524 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:04.173961 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0573bd0e-c1cd-4b9f-8395-194fdccd3524 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:04.174398 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0573bd0e-c1cd-4b9f-8395-194fdccd3524 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:04.251546 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-0573bd0e-c1cd-4b9f-8395-194fdccd3524 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:04.264991 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-0573bd0e-c1cd-4b9f-8395-194fdccd3524 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3" status: 200 len: 1301 microversion: 2.1 time: 0.108313 Aug 08 02:31:04.278211 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 374/745] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:31:04 2022] GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3 => generated 1301 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:31:05.292563 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-d68fbb9d-0295-42c7-ae68-ab124081e839 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:05.298628 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-d68fbb9d-0295-42c7-ae68-ab124081e839 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: cc2b42f1-3dd0-4090-a905-93283047aea3] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:05.299802 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d68fbb9d-0295-42c7-ae68-ab124081e839 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:05.300352 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d68fbb9d-0295-42c7-ae68-ab124081e839 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:05.300995 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d68fbb9d-0295-42c7-ae68-ab124081e839 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:05.301691 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d68fbb9d-0295-42c7-ae68-ab124081e839 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:05.302216 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d68fbb9d-0295-42c7-ae68-ab124081e839 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:05.302537 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d68fbb9d-0295-42c7-ae68-ab124081e839 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:05.465224 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-d68fbb9d-0295-42c7-ae68-ab124081e839 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:05.473675 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-d68fbb9d-0295-42c7-ae68-ab124081e839 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3" status: 200 len: 1366 microversion: 2.1 time: 0.185655 Aug 08 02:31:05.474827 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 372/746] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:31:05 2022] GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3 => generated 1366 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:31:06.109775 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [req-c5716d48-f067-4bfb-8fbc-8302f09ab429 req-fd19bddb-20bc-4f14-9e8c-6eaab10f25d7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "cc2b42f1-3dd0-4090-a905-93283047aea3", "tag": "77fffbdc-6617-43da-9c07-c8e3bd46cfe3"}]} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:31:06.130699 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-c5716d48-f067-4bfb-8fbc-8302f09ab429 req-fd19bddb-20bc-4f14-9e8c-6eaab10f25d7 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:06.131149 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-c5716d48-f067-4bfb-8fbc-8302f09ab429 req-fd19bddb-20bc-4f14-9e8c-6eaab10f25d7 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:06.131970 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-c5716d48-f067-4bfb-8fbc-8302f09ab429 req-fd19bddb-20bc-4f14-9e8c-6eaab10f25d7 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:06.149189 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.compute.server_external_events [req-c5716d48-f067-4bfb-8fbc-8302f09ab429 req-fd19bddb-20bc-4f14-9e8c-6eaab10f25d7 service nova] Creating event network-changed:77fffbdc-6617-43da-9c07-c8e3bd46cfe3 for instance cc2b42f1-3dd0-4090-a905-93283047aea3 on hv-cinder-74014 Aug 08 02:31:06.155685 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [req-c5716d48-f067-4bfb-8fbc-8302f09ab429 req-fd19bddb-20bc-4f14-9e8c-6eaab10f25d7 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.049195 Aug 08 02:31:06.156767 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 375/747] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:31:06 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:31:06.495568 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-3cdb2f93-ebf7-4d17-a6fc-31d84eab7ad8 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:06.496347 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-3cdb2f93-ebf7-4d17-a6fc-31d84eab7ad8 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: cc2b42f1-3dd0-4090-a905-93283047aea3] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:06.503389 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3cdb2f93-ebf7-4d17-a6fc-31d84eab7ad8 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:06.504080 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3cdb2f93-ebf7-4d17-a6fc-31d84eab7ad8 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:06.504761 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3cdb2f93-ebf7-4d17-a6fc-31d84eab7ad8 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:06.505909 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3cdb2f93-ebf7-4d17-a6fc-31d84eab7ad8 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:06.506566 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3cdb2f93-ebf7-4d17-a6fc-31d84eab7ad8 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:06.507186 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3cdb2f93-ebf7-4d17-a6fc-31d84eab7ad8 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:06.659925 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-3cdb2f93-ebf7-4d17-a6fc-31d84eab7ad8 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:06.668539 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-3cdb2f93-ebf7-4d17-a6fc-31d84eab7ad8 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3" status: 200 len: 1366 microversion: 2.1 time: 0.177269 Aug 08 02:31:06.673924 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 373/748] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:31:06 2022] GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3 => generated 1366 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:31:07.693378 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-988b5c7f-1284-4c09-8c93-daddd3745f7e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:07.694004 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-988b5c7f-1284-4c09-8c93-daddd3745f7e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: cc2b42f1-3dd0-4090-a905-93283047aea3] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:07.701208 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-988b5c7f-1284-4c09-8c93-daddd3745f7e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:07.701696 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-988b5c7f-1284-4c09-8c93-daddd3745f7e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:07.702135 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-988b5c7f-1284-4c09-8c93-daddd3745f7e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:07.702848 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-988b5c7f-1284-4c09-8c93-daddd3745f7e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:07.703351 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-988b5c7f-1284-4c09-8c93-daddd3745f7e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:07.703782 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-988b5c7f-1284-4c09-8c93-daddd3745f7e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:07.838635 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-988b5c7f-1284-4c09-8c93-daddd3745f7e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:07.846277 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-988b5c7f-1284-4c09-8c93-daddd3745f7e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3" status: 200 len: 1366 microversion: 2.1 time: 0.157441 Aug 08 02:31:07.846850 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 376/749] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:31:07 2022] GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3 => generated 1366 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:31:08.867940 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-fe15bb02-5cad-4194-8ebc-f7d0276186e1 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:08.868380 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-fe15bb02-5cad-4194-8ebc-f7d0276186e1 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: cc2b42f1-3dd0-4090-a905-93283047aea3] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:08.875915 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-fe15bb02-5cad-4194-8ebc-f7d0276186e1 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:08.876395 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-fe15bb02-5cad-4194-8ebc-f7d0276186e1 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:08.876868 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-fe15bb02-5cad-4194-8ebc-f7d0276186e1 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:08.877568 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-fe15bb02-5cad-4194-8ebc-f7d0276186e1 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:08.878036 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-fe15bb02-5cad-4194-8ebc-f7d0276186e1 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:08.878453 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-fe15bb02-5cad-4194-8ebc-f7d0276186e1 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:09.014310 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-fe15bb02-5cad-4194-8ebc-f7d0276186e1 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:09.021815 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-fe15bb02-5cad-4194-8ebc-f7d0276186e1 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3" status: 200 len: 1366 microversion: 2.1 time: 0.157932 Aug 08 02:31:09.022383 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 374/750] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:31:08 2022] GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3 => generated 1366 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:31:09.601238 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-9ae416d5-80de-42e9-be82-5f6f2efe2bdd tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:09.601718 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-9ae416d5-80de-42e9-be82-5f6f2efe2bdd tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:09.608734 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9ae416d5-80de-42e9-be82-5f6f2efe2bdd tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:09.609254 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9ae416d5-80de-42e9-be82-5f6f2efe2bdd tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:09.609721 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9ae416d5-80de-42e9-be82-5f6f2efe2bdd tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:09.610415 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9ae416d5-80de-42e9-be82-5f6f2efe2bdd tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:09.610877 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9ae416d5-80de-42e9-be82-5f6f2efe2bdd tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:09.611305 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9ae416d5-80de-42e9-be82-5f6f2efe2bdd tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:09.634982 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-9ae416d5-80de-42e9-be82-5f6f2efe2bdd tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d/ips" status: 200 len: 152 microversion: 2.20 time: 0.205964 Aug 08 02:31:09.635540 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 377/751] 10.222.0.27 () {60 vars in 1289 bytes} [Mon Aug 8 02:31:09 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d/ips => generated 152 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 08 02:31:10.042992 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-98a64993-2c4a-4099-a70f-25d605412f84 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:10.043616 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-98a64993-2c4a-4099-a70f-25d605412f84 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: cc2b42f1-3dd0-4090-a905-93283047aea3] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:10.050060 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-98a64993-2c4a-4099-a70f-25d605412f84 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:10.050544 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-98a64993-2c4a-4099-a70f-25d605412f84 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:10.050978 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-98a64993-2c4a-4099-a70f-25d605412f84 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:10.051655 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-98a64993-2c4a-4099-a70f-25d605412f84 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:10.052105 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-98a64993-2c4a-4099-a70f-25d605412f84 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:10.052770 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-98a64993-2c4a-4099-a70f-25d605412f84 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:10.189456 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-98a64993-2c4a-4099-a70f-25d605412f84 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:10.197720 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-98a64993-2c4a-4099-a70f-25d605412f84 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3" status: 200 len: 1366 microversion: 2.1 time: 0.158195 Aug 08 02:31:10.198893 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 375/752] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:31:10 2022] GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3 => generated 1366 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:31:11.219955 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-d159fe94-0afb-46a9-964f-024ad48b9c8a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:11.220613 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-d159fe94-0afb-46a9-964f-024ad48b9c8a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: cc2b42f1-3dd0-4090-a905-93283047aea3] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:11.223423 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell0:SELECT=2 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:11.231473 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d159fe94-0afb-46a9-964f-024ad48b9c8a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:11.231970 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d159fe94-0afb-46a9-964f-024ad48b9c8a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:11.232416 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d159fe94-0afb-46a9-964f-024ad48b9c8a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:11.233156 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d159fe94-0afb-46a9-964f-024ad48b9c8a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:11.233598 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d159fe94-0afb-46a9-964f-024ad48b9c8a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:11.234057 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d159fe94-0afb-46a9-964f-024ad48b9c8a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:11.371120 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-d159fe94-0afb-46a9-964f-024ad48b9c8a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:11.379253 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-d159fe94-0afb-46a9-964f-024ad48b9c8a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3" status: 200 len: 1366 microversion: 2.1 time: 0.163146 Aug 08 02:31:11.379851 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 378/753] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:31:11 2022] GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3 => generated 1366 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:31:12.323884 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell0:SELECT=6 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:12.380628 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [req-6fd5c9b0-f839-4ea9-a4f9-03e2727f40c8 req-23abb707-b885-4e09-9c94-057ccb8a29b3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cc2b42f1-3dd0-4090-a905-93283047aea3", "name": "network-vif-plugged", "status": "completed", "tag": "77fffbdc-6617-43da-9c07-c8e3bd46cfe3"}]} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:31:12.392638 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-85eba746-f97e-4f37-b0aa-5b4eacbe8a3d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:31:12.401133 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-85eba746-f97e-4f37-b0aa-5b4eacbe8a3d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:12.401320 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-6fd5c9b0-f839-4ea9-a4f9-03e2727f40c8 req-23abb707-b885-4e09-9c94-057ccb8a29b3 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:12.401320 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-6fd5c9b0-f839-4ea9-a4f9-03e2727f40c8 req-23abb707-b885-4e09-9c94-057ccb8a29b3 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:12.401510 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell0:SELECT=8 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:12.401626 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-6fd5c9b0-f839-4ea9-a4f9-03e2727f40c8 req-23abb707-b885-4e09-9c94-057ccb8a29b3 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:12.401714 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_api:SELECT=82,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:12.401714 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-85eba746-f97e-4f37-b0aa-5b4eacbe8a3d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:12.402197 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-85eba746-f97e-4f37-b0aa-5b4eacbe8a3d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:12.402636 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-85eba746-f97e-4f37-b0aa-5b4eacbe8a3d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:12.403354 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-85eba746-f97e-4f37-b0aa-5b4eacbe8a3d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:12.403814 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-85eba746-f97e-4f37-b0aa-5b4eacbe8a3d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:12.404241 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-85eba746-f97e-4f37-b0aa-5b4eacbe8a3d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:12.411435 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.compute.server_external_events [req-6fd5c9b0-f839-4ea9-a4f9-03e2727f40c8 req-23abb707-b885-4e09-9c94-057ccb8a29b3 service nova] Creating event network-vif-plugged:77fffbdc-6617-43da-9c07-c8e3bd46cfe3 for instance cc2b42f1-3dd0-4090-a905-93283047aea3 on hv-cinder-74014 Aug 08 02:31:12.418749 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [req-6fd5c9b0-f839-4ea9-a4f9-03e2727f40c8 req-23abb707-b885-4e09-9c94-057ccb8a29b3 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.114438 Aug 08 02:31:12.421774 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 376/754] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:31:12 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:31:12.429857 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.objects.instance [None req-85eba746-f97e-4f37-b0aa-5b4eacbe8a3d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lazy-loading 'flavor' on Instance uuid 946fd571-bd98-44f9-89f4-52f0116b642d {{(pid=118352) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 08 02:31:12.446469 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-2af59f06-a951-471a-bfd1-b5fd35946c92 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:12.447226 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-2af59f06-a951-471a-bfd1-b5fd35946c92 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: cc2b42f1-3dd0-4090-a905-93283047aea3] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:12.455038 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2af59f06-a951-471a-bfd1-b5fd35946c92 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:12.455519 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2af59f06-a951-471a-bfd1-b5fd35946c92 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:12.456271 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2af59f06-a951-471a-bfd1-b5fd35946c92 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:12.457278 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2af59f06-a951-471a-bfd1-b5fd35946c92 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:12.457957 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2af59f06-a951-471a-bfd1-b5fd35946c92 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:12.458370 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2af59f06-a951-471a-bfd1-b5fd35946c92 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:12.608429 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-2af59f06-a951-471a-bfd1-b5fd35946c92 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:12.616186 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-2af59f06-a951-471a-bfd1-b5fd35946c92 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3" status: 200 len: 1366 microversion: 2.1 time: 0.181854 Aug 08 02:31:12.620760 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 377/755] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:31:12 2022] GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3 => generated 1366 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:31:13.190872 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-85eba746-f97e-4f37-b0aa-5b4eacbe8a3d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "POST /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d/action" status: 202 len: 0 microversion: 2.20 time: 0.802243 Aug 08 02:31:13.191455 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 379/756] 10.222.0.27 () {62 vars in 1319 bytes} [Mon Aug 8 02:31:12 2022] POST /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d/action => generated 0 bytes in 804 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Aug 08 02:31:13.200296 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-960ded43-3902-4e45-922c-b1869f85a41e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:13.200937 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-960ded43-3902-4e45-922c-b1869f85a41e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:13.206833 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-960ded43-3902-4e45-922c-b1869f85a41e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:13.207312 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-960ded43-3902-4e45-922c-b1869f85a41e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:13.207742 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-960ded43-3902-4e45-922c-b1869f85a41e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:13.208442 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-960ded43-3902-4e45-922c-b1869f85a41e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:13.208920 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-960ded43-3902-4e45-922c-b1869f85a41e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:13.209342 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-960ded43-3902-4e45-922c-b1869f85a41e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:13.348947 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-960ded43-3902-4e45-922c-b1869f85a41e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:13.357309 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-960ded43-3902-4e45-922c-b1869f85a41e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:13.358788 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-960ded43-3902-4e45-922c-b1869f85a41e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:13.360007 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-960ded43-3902-4e45-922c-b1869f85a41e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1723 microversion: 2.20 time: 0.164019 Aug 08 02:31:13.360560 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 378/757] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:31:13 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1723 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:31:13.638721 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-a35761f1-d2f3-4524-bd3a-9028f8db4fbb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:13.639462 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-a35761f1-d2f3-4524-bd3a-9028f8db4fbb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: cc2b42f1-3dd0-4090-a905-93283047aea3] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:13.655692 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a35761f1-d2f3-4524-bd3a-9028f8db4fbb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:13.655692 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a35761f1-d2f3-4524-bd3a-9028f8db4fbb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:13.655692 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a35761f1-d2f3-4524-bd3a-9028f8db4fbb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:13.655692 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a35761f1-d2f3-4524-bd3a-9028f8db4fbb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:13.656125 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a35761f1-d2f3-4524-bd3a-9028f8db4fbb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:13.656125 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a35761f1-d2f3-4524-bd3a-9028f8db4fbb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:13.790546 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-a35761f1-d2f3-4524-bd3a-9028f8db4fbb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:13.798005 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-a35761f1-d2f3-4524-bd3a-9028f8db4fbb tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3" status: 200 len: 1366 microversion: 2.1 time: 0.165381 Aug 08 02:31:13.798616 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 380/758] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:31:13 2022] GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3 => generated 1366 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:31:14.383447 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-1ebd9bbe-c2ec-44a9-9afb-844c7fa8e99a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:14.384046 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-1ebd9bbe-c2ec-44a9-9afb-844c7fa8e99a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:14.392101 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1ebd9bbe-c2ec-44a9-9afb-844c7fa8e99a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:14.392239 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1ebd9bbe-c2ec-44a9-9afb-844c7fa8e99a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:14.392625 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1ebd9bbe-c2ec-44a9-9afb-844c7fa8e99a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:14.393336 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1ebd9bbe-c2ec-44a9-9afb-844c7fa8e99a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:14.393931 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1ebd9bbe-c2ec-44a9-9afb-844c7fa8e99a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:14.394209 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1ebd9bbe-c2ec-44a9-9afb-844c7fa8e99a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:14.529336 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-1ebd9bbe-c2ec-44a9-9afb-844c7fa8e99a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:14.537759 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-1ebd9bbe-c2ec-44a9-9afb-844c7fa8e99a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:14.539272 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-1ebd9bbe-c2ec-44a9-9afb-844c7fa8e99a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:14.553132 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-1ebd9bbe-c2ec-44a9-9afb-844c7fa8e99a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1723 microversion: 2.20 time: 0.160897 Aug 08 02:31:14.553132 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 379/759] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:31:14 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1723 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:31:14.818973 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-3ec1b64f-c09c-499b-989c-1b87921604ac tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:14.819582 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-3ec1b64f-c09c-499b-989c-1b87921604ac tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: cc2b42f1-3dd0-4090-a905-93283047aea3] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:14.827215 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3ec1b64f-c09c-499b-989c-1b87921604ac tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:14.827696 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3ec1b64f-c09c-499b-989c-1b87921604ac tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:14.828133 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3ec1b64f-c09c-499b-989c-1b87921604ac tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:14.828848 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3ec1b64f-c09c-499b-989c-1b87921604ac tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:14.829307 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3ec1b64f-c09c-499b-989c-1b87921604ac tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:14.829846 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3ec1b64f-c09c-499b-989c-1b87921604ac tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:14.965706 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-3ec1b64f-c09c-499b-989c-1b87921604ac tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:14.973449 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-3ec1b64f-c09c-499b-989c-1b87921604ac tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3" status: 200 len: 1366 microversion: 2.1 time: 0.158290 Aug 08 02:31:14.975253 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 381/760] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:31:14 2022] GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3 => generated 1366 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:31:15.564908 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-96c9c31e-ed4d-41a1-b4f9-1baaedd84379 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:15.565548 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-96c9c31e-ed4d-41a1-b4f9-1baaedd84379 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:15.572267 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-96c9c31e-ed4d-41a1-b4f9-1baaedd84379 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:15.572782 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-96c9c31e-ed4d-41a1-b4f9-1baaedd84379 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:15.573220 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-96c9c31e-ed4d-41a1-b4f9-1baaedd84379 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:15.573912 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-96c9c31e-ed4d-41a1-b4f9-1baaedd84379 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:15.574367 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-96c9c31e-ed4d-41a1-b4f9-1baaedd84379 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:15.574793 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-96c9c31e-ed4d-41a1-b4f9-1baaedd84379 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:15.710995 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-96c9c31e-ed4d-41a1-b4f9-1baaedd84379 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:15.720518 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-96c9c31e-ed4d-41a1-b4f9-1baaedd84379 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:15.722113 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-96c9c31e-ed4d-41a1-b4f9-1baaedd84379 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:15.723277 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-96c9c31e-ed4d-41a1-b4f9-1baaedd84379 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1723 microversion: 2.20 time: 0.162481 Aug 08 02:31:15.723848 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 380/761] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:31:15 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1723 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:31:15.995476 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-092f7397-7ef7-494a-894a-154616157ba8 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:15.995476 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-092f7397-7ef7-494a-894a-154616157ba8 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: cc2b42f1-3dd0-4090-a905-93283047aea3] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:16.001728 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-092f7397-7ef7-494a-894a-154616157ba8 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:16.002217 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-092f7397-7ef7-494a-894a-154616157ba8 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:16.002658 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-092f7397-7ef7-494a-894a-154616157ba8 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:16.003359 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-092f7397-7ef7-494a-894a-154616157ba8 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:16.003819 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-092f7397-7ef7-494a-894a-154616157ba8 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:16.004250 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-092f7397-7ef7-494a-894a-154616157ba8 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:16.137719 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-092f7397-7ef7-494a-894a-154616157ba8 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:16.145252 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-092f7397-7ef7-494a-894a-154616157ba8 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3" status: 200 len: 1557 microversion: 2.1 time: 0.155467 Aug 08 02:31:16.145808 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 382/762] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:31:15 2022] GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3 => generated 1557 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:31:16.166533 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-3d68a7b1-0e5c-4f07-a96f-6c095d157a64 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:16.167133 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-3d68a7b1-0e5c-4f07-a96f-6c095d157a64 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: cc2b42f1-3dd0-4090-a905-93283047aea3] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:16.173183 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3d68a7b1-0e5c-4f07-a96f-6c095d157a64 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:16.174477 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3d68a7b1-0e5c-4f07-a96f-6c095d157a64 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:16.174477 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3d68a7b1-0e5c-4f07-a96f-6c095d157a64 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:16.174766 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3d68a7b1-0e5c-4f07-a96f-6c095d157a64 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:16.175219 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3d68a7b1-0e5c-4f07-a96f-6c095d157a64 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:16.175696 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3d68a7b1-0e5c-4f07-a96f-6c095d157a64 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:16.258867 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-3d68a7b1-0e5c-4f07-a96f-6c095d157a64 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.097013 Aug 08 02:31:16.259490 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 381/763] 10.222.0.27 () {58 vars in 1275 bytes} [Mon Aug 8 02:31:16 2022] GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3/os-interface => generated 285 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:31:16.748053 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-097806ac-38e5-4042-9c36-914e40e70b10 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:16.748696 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-097806ac-38e5-4042-9c36-914e40e70b10 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:16.755005 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-097806ac-38e5-4042-9c36-914e40e70b10 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:16.755514 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-097806ac-38e5-4042-9c36-914e40e70b10 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:16.755952 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-097806ac-38e5-4042-9c36-914e40e70b10 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:16.756640 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-097806ac-38e5-4042-9c36-914e40e70b10 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:16.757121 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-097806ac-38e5-4042-9c36-914e40e70b10 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:16.757582 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-097806ac-38e5-4042-9c36-914e40e70b10 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:16.986450 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-097806ac-38e5-4042-9c36-914e40e70b10 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:16.995748 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-097806ac-38e5-4042-9c36-914e40e70b10 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:16.997307 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-097806ac-38e5-4042-9c36-914e40e70b10 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:16.998520 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-097806ac-38e5-4042-9c36-914e40e70b10 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1723 microversion: 2.20 time: 0.255100 Aug 08 02:31:17.012693 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 383/764] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:31:16 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1723 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:31:17.678353 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [req-da0a4033-9cec-4e35-b8d6-ea0b7aa9cca7 req-a017895e-b429-42f2-97f8-8df661c8f20e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "cc2b42f1-3dd0-4090-a905-93283047aea3", "tag": "77fffbdc-6617-43da-9c07-c8e3bd46cfe3"}]} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:31:17.688339 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-da0a4033-9cec-4e35-b8d6-ea0b7aa9cca7 req-a017895e-b429-42f2-97f8-8df661c8f20e service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:17.688853 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-da0a4033-9cec-4e35-b8d6-ea0b7aa9cca7 req-a017895e-b429-42f2-97f8-8df661c8f20e service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:17.689286 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-da0a4033-9cec-4e35-b8d6-ea0b7aa9cca7 req-a017895e-b429-42f2-97f8-8df661c8f20e service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:17.702638 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.compute.server_external_events [req-da0a4033-9cec-4e35-b8d6-ea0b7aa9cca7 req-a017895e-b429-42f2-97f8-8df661c8f20e service nova] Creating event network-changed:77fffbdc-6617-43da-9c07-c8e3bd46cfe3 for instance cc2b42f1-3dd0-4090-a905-93283047aea3 on hv-cinder-74014 Aug 08 02:31:17.706387 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [req-da0a4033-9cec-4e35-b8d6-ea0b7aa9cca7 req-a017895e-b429-42f2-97f8-8df661c8f20e service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.089678 Aug 08 02:31:17.706995 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 382/765] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:31:17 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:31:18.021351 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-0c531282-6125-4687-bc8f-5aa390222270 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:18.021961 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-0c531282-6125-4687-bc8f-5aa390222270 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:18.028525 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0c531282-6125-4687-bc8f-5aa390222270 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:18.029043 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0c531282-6125-4687-bc8f-5aa390222270 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:18.029475 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0c531282-6125-4687-bc8f-5aa390222270 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:18.030165 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0c531282-6125-4687-bc8f-5aa390222270 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:18.030623 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0c531282-6125-4687-bc8f-5aa390222270 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:18.031044 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0c531282-6125-4687-bc8f-5aa390222270 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:18.180292 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-0c531282-6125-4687-bc8f-5aa390222270 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:18.190210 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-0c531282-6125-4687-bc8f-5aa390222270 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:18.194414 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-0c531282-6125-4687-bc8f-5aa390222270 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:18.194414 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-0c531282-6125-4687-bc8f-5aa390222270 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1723 microversion: 2.20 time: 0.175340 Aug 08 02:31:18.194414 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 384/766] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:31:18 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1723 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:31:19.218631 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-52e5d72d-847a-4b8a-8f99-58728517bebf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:19.219500 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-52e5d72d-847a-4b8a-8f99-58728517bebf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:19.225786 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-52e5d72d-847a-4b8a-8f99-58728517bebf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:19.227652 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-52e5d72d-847a-4b8a-8f99-58728517bebf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:19.227652 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-52e5d72d-847a-4b8a-8f99-58728517bebf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:19.227652 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-52e5d72d-847a-4b8a-8f99-58728517bebf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:19.228034 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-52e5d72d-847a-4b8a-8f99-58728517bebf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:19.228272 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-52e5d72d-847a-4b8a-8f99-58728517bebf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:19.369874 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-52e5d72d-847a-4b8a-8f99-58728517bebf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:19.378719 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-52e5d72d-847a-4b8a-8f99-58728517bebf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:19.380361 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-52e5d72d-847a-4b8a-8f99-58728517bebf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:19.381545 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-52e5d72d-847a-4b8a-8f99-58728517bebf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1723 microversion: 2.20 time: 0.168362 Aug 08 02:31:19.382148 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 383/767] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:31:19 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1723 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:31:20.406195 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-5f8ddd5a-4144-4d84-ad7a-bbe04af9f507 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:20.406812 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-5f8ddd5a-4144-4d84-ad7a-bbe04af9f507 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:20.413505 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5f8ddd5a-4144-4d84-ad7a-bbe04af9f507 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:20.413990 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5f8ddd5a-4144-4d84-ad7a-bbe04af9f507 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:20.414428 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5f8ddd5a-4144-4d84-ad7a-bbe04af9f507 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:20.415503 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5f8ddd5a-4144-4d84-ad7a-bbe04af9f507 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:20.415884 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5f8ddd5a-4144-4d84-ad7a-bbe04af9f507 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:20.416342 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5f8ddd5a-4144-4d84-ad7a-bbe04af9f507 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:20.549696 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-5f8ddd5a-4144-4d84-ad7a-bbe04af9f507 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:20.557799 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-5f8ddd5a-4144-4d84-ad7a-bbe04af9f507 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:20.560045 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-5f8ddd5a-4144-4d84-ad7a-bbe04af9f507 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:20.561222 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-5f8ddd5a-4144-4d84-ad7a-bbe04af9f507 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1723 microversion: 2.20 time: 0.159590 Aug 08 02:31:20.561809 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 385/768] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:31:20 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1723 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:31:21.586544 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-92f9256a-e1d1-4402-ae4e-cd799bfe53f7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:21.587360 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-92f9256a-e1d1-4402-ae4e-cd799bfe53f7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:21.590410 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_api:SELECT=64,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:21.595543 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-92f9256a-e1d1-4402-ae4e-cd799bfe53f7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:21.598182 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-92f9256a-e1d1-4402-ae4e-cd799bfe53f7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:21.598182 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-92f9256a-e1d1-4402-ae4e-cd799bfe53f7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:21.598182 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-92f9256a-e1d1-4402-ae4e-cd799bfe53f7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:21.598182 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-92f9256a-e1d1-4402-ae4e-cd799bfe53f7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:21.598562 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-92f9256a-e1d1-4402-ae4e-cd799bfe53f7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:21.733834 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-92f9256a-e1d1-4402-ae4e-cd799bfe53f7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:21.741762 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-92f9256a-e1d1-4402-ae4e-cd799bfe53f7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:21.743254 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-92f9256a-e1d1-4402-ae4e-cd799bfe53f7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:21.744397 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-92f9256a-e1d1-4402-ae4e-cd799bfe53f7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1723 microversion: 2.20 time: 0.162066 Aug 08 02:31:21.744984 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 384/769] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:31:21 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1723 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:31:22.772659 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-ce2cff1c-8e4b-42ee-931d-f66663e03e0e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:22.773286 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-ce2cff1c-8e4b-42ee-931d-f66663e03e0e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:22.779892 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ce2cff1c-8e4b-42ee-931d-f66663e03e0e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:22.780357 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ce2cff1c-8e4b-42ee-931d-f66663e03e0e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:22.780822 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ce2cff1c-8e4b-42ee-931d-f66663e03e0e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:22.781503 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ce2cff1c-8e4b-42ee-931d-f66663e03e0e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:22.781954 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ce2cff1c-8e4b-42ee-931d-f66663e03e0e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:22.782376 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ce2cff1c-8e4b-42ee-931d-f66663e03e0e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:22.911650 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-ce2cff1c-8e4b-42ee-931d-f66663e03e0e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:22.920271 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-ce2cff1c-8e4b-42ee-931d-f66663e03e0e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:22.921790 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-ce2cff1c-8e4b-42ee-931d-f66663e03e0e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:22.922929 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-ce2cff1c-8e4b-42ee-931d-f66663e03e0e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1723 microversion: 2.20 time: 0.154230 Aug 08 02:31:22.923508 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 386/770] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:31:22 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1723 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:31:23.946224 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-5adc43f9-cc47-46fa-a1ce-fc50e49b1554 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:23.950660 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-5adc43f9-cc47-46fa-a1ce-fc50e49b1554 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:23.953865 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5adc43f9-cc47-46fa-a1ce-fc50e49b1554 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:23.954553 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5adc43f9-cc47-46fa-a1ce-fc50e49b1554 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:23.954994 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5adc43f9-cc47-46fa-a1ce-fc50e49b1554 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:23.955701 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5adc43f9-cc47-46fa-a1ce-fc50e49b1554 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:23.956157 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5adc43f9-cc47-46fa-a1ce-fc50e49b1554 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:23.956585 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5adc43f9-cc47-46fa-a1ce-fc50e49b1554 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:24.094850 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-5adc43f9-cc47-46fa-a1ce-fc50e49b1554 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:24.103040 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-5adc43f9-cc47-46fa-a1ce-fc50e49b1554 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:24.104587 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-5adc43f9-cc47-46fa-a1ce-fc50e49b1554 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:24.106281 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-5adc43f9-cc47-46fa-a1ce-fc50e49b1554 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1723 microversion: 2.20 time: 0.163755 Aug 08 02:31:24.106989 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 385/771] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:31:23 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1723 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:31:24.656247 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-e3d5886d-521c-4982-87a6-476821f7d80d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:24.656809 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-e3d5886d-521c-4982-87a6-476821f7d80d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: 48b3d4bf-6b20-4327-8f8d-521417405254] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:24.663597 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e3d5886d-521c-4982-87a6-476821f7d80d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:24.664075 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e3d5886d-521c-4982-87a6-476821f7d80d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:24.664503 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e3d5886d-521c-4982-87a6-476821f7d80d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:24.665290 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e3d5886d-521c-4982-87a6-476821f7d80d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:24.665737 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e3d5886d-521c-4982-87a6-476821f7d80d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:24.666158 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e3d5886d-521c-4982-87a6-476821f7d80d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:24.670501 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell1:SELECT=92,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:24.690071 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-e3d5886d-521c-4982-87a6-476821f7d80d tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254/ips" status: 200 len: 145 microversion: 2.1 time: 0.038182 Aug 08 02:31:24.690648 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 387/772] 10.222.0.27 () {58 vars in 1248 bytes} [Mon Aug 8 02:31:24 2022] GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254/ips => generated 145 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:31:25.129319 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-282dd8bc-d713-4a92-8c34-9a93a8a1e4ae tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:25.129928 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-282dd8bc-d713-4a92-8c34-9a93a8a1e4ae tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:25.136075 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-282dd8bc-d713-4a92-8c34-9a93a8a1e4ae tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:25.136561 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-282dd8bc-d713-4a92-8c34-9a93a8a1e4ae tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:25.137392 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-282dd8bc-d713-4a92-8c34-9a93a8a1e4ae tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:25.137802 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-282dd8bc-d713-4a92-8c34-9a93a8a1e4ae tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:25.138258 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-282dd8bc-d713-4a92-8c34-9a93a8a1e4ae tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:25.138714 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-282dd8bc-d713-4a92-8c34-9a93a8a1e4ae tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:25.275993 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-282dd8bc-d713-4a92-8c34-9a93a8a1e4ae tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:25.284874 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-282dd8bc-d713-4a92-8c34-9a93a8a1e4ae tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:25.286512 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-282dd8bc-d713-4a92-8c34-9a93a8a1e4ae tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:25.287675 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-282dd8bc-d713-4a92-8c34-9a93a8a1e4ae tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1723 microversion: 2.20 time: 0.162265 Aug 08 02:31:25.288224 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 386/773] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:31:25 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1723 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:31:26.311839 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-bda18d7f-c4fc-4201-a246-04645e0e734c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:26.312453 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-bda18d7f-c4fc-4201-a246-04645e0e734c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:26.319378 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-bda18d7f-c4fc-4201-a246-04645e0e734c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:26.319863 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-bda18d7f-c4fc-4201-a246-04645e0e734c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:26.320558 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-bda18d7f-c4fc-4201-a246-04645e0e734c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:26.321283 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-bda18d7f-c4fc-4201-a246-04645e0e734c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:26.321740 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-bda18d7f-c4fc-4201-a246-04645e0e734c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:26.322167 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-bda18d7f-c4fc-4201-a246-04645e0e734c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:26.396008 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-d6944c71-3a5d-4306-9707-cc7d13d6cd0e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "e89ba9d3-726f-4c26-8f44-1e0b7d17a2b3"}} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:31:26.398351 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-d6944c71-3a5d-4306-9707-cc7d13d6cd0e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: 48b3d4bf-6b20-4327-8f8d-521417405254] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:26.405067 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d6944c71-3a5d-4306-9707-cc7d13d6cd0e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:26.405775 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d6944c71-3a5d-4306-9707-cc7d13d6cd0e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:26.406224 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d6944c71-3a5d-4306-9707-cc7d13d6cd0e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:26.406972 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d6944c71-3a5d-4306-9707-cc7d13d6cd0e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:26.407436 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d6944c71-3a5d-4306-9707-cc7d13d6cd0e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:26.407885 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d6944c71-3a5d-4306-9707-cc7d13d6cd0e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:26.467424 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-bda18d7f-c4fc-4201-a246-04645e0e734c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:26.476234 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-bda18d7f-c4fc-4201-a246-04645e0e734c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:26.477884 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-bda18d7f-c4fc-4201-a246-04645e0e734c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:26.479042 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-bda18d7f-c4fc-4201-a246-04645e0e734c tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1723 microversion: 2.20 time: 0.172144 Aug 08 02:31:26.479590 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 388/774] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:31:26 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1723 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 08 02:31:26.483955 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell1:SELECT=100 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:27.177830 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-d6944c71-3a5d-4306-9707-cc7d13d6cd0e tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "POST /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.785522 Aug 08 02:31:27.178372 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 387/775] 10.222.0.27 () {60 vars in 1323 bytes} [Mon Aug 8 02:31:26 2022] POST /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254/os-volume_attachments => generated 194 bytes in 787 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 08 02:31:27.503368 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-b2c659dd-da04-4114-9a64-37b9c50d3c98 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:27.504002 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-b2c659dd-da04-4114-9a64-37b9c50d3c98 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:27.511088 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b2c659dd-da04-4114-9a64-37b9c50d3c98 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:27.511577 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b2c659dd-da04-4114-9a64-37b9c50d3c98 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:27.512781 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b2c659dd-da04-4114-9a64-37b9c50d3c98 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:27.513488 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b2c659dd-da04-4114-9a64-37b9c50d3c98 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:27.513948 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b2c659dd-da04-4114-9a64-37b9c50d3c98 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:27.514376 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b2c659dd-da04-4114-9a64-37b9c50d3c98 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:27.651742 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-b2c659dd-da04-4114-9a64-37b9c50d3c98 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:27.660101 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-b2c659dd-da04-4114-9a64-37b9c50d3c98 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:27.661639 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-b2c659dd-da04-4114-9a64-37b9c50d3c98 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:27.662809 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-b2c659dd-da04-4114-9a64-37b9c50d3c98 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1723 microversion: 2.20 time: 0.163715 Aug 08 02:31:27.663379 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 389/776] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:31:27 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1723 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:31:28.686877 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-10f3d05c-9822-45f2-98d4-e0e8f588879f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:28.687570 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-10f3d05c-9822-45f2-98d4-e0e8f588879f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:28.693810 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-10f3d05c-9822-45f2-98d4-e0e8f588879f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:28.694295 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-10f3d05c-9822-45f2-98d4-e0e8f588879f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:28.694729 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-10f3d05c-9822-45f2-98d4-e0e8f588879f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:28.695682 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-10f3d05c-9822-45f2-98d4-e0e8f588879f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:28.696135 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-10f3d05c-9822-45f2-98d4-e0e8f588879f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:28.696560 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-10f3d05c-9822-45f2-98d4-e0e8f588879f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:28.825982 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-10f3d05c-9822-45f2-98d4-e0e8f588879f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:28.834137 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-10f3d05c-9822-45f2-98d4-e0e8f588879f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:28.835647 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-10f3d05c-9822-45f2-98d4-e0e8f588879f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:28.836836 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-10f3d05c-9822-45f2-98d4-e0e8f588879f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1723 microversion: 2.20 time: 0.154800 Aug 08 02:31:28.837453 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 388/777] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:31:28 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1723 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:31:29.860178 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-75f5654d-c0a4-47e3-a8c5-a7f65d9a76d3 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:29.860831 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-75f5654d-c0a4-47e3-a8c5-a7f65d9a76d3 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:29.867274 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-75f5654d-c0a4-47e3-a8c5-a7f65d9a76d3 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:29.867759 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-75f5654d-c0a4-47e3-a8c5-a7f65d9a76d3 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:29.868199 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-75f5654d-c0a4-47e3-a8c5-a7f65d9a76d3 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:29.868922 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-75f5654d-c0a4-47e3-a8c5-a7f65d9a76d3 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:29.869379 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-75f5654d-c0a4-47e3-a8c5-a7f65d9a76d3 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:29.869807 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-75f5654d-c0a4-47e3-a8c5-a7f65d9a76d3 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:30.006010 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-75f5654d-c0a4-47e3-a8c5-a7f65d9a76d3 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:30.014560 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-75f5654d-c0a4-47e3-a8c5-a7f65d9a76d3 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:30.017289 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-75f5654d-c0a4-47e3-a8c5-a7f65d9a76d3 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:30.017289 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-75f5654d-c0a4-47e3-a8c5-a7f65d9a76d3 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1723 microversion: 2.20 time: 0.160975 Aug 08 02:31:30.017772 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 390/778] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:31:29 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1723 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:31:31.040589 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-20e12f79-389d-4b21-8ed1-73da405e0232 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:31.041232 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-20e12f79-389d-4b21-8ed1-73da405e0232 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:31.048130 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-20e12f79-389d-4b21-8ed1-73da405e0232 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:31.048619 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-20e12f79-389d-4b21-8ed1-73da405e0232 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:31.058690 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-20e12f79-389d-4b21-8ed1-73da405e0232 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:31.058690 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-20e12f79-389d-4b21-8ed1-73da405e0232 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:31.058690 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-20e12f79-389d-4b21-8ed1-73da405e0232 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:31.058690 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-20e12f79-389d-4b21-8ed1-73da405e0232 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:31.182437 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-20e12f79-389d-4b21-8ed1-73da405e0232 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:31.191068 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-20e12f79-389d-4b21-8ed1-73da405e0232 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:31.192560 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-20e12f79-389d-4b21-8ed1-73da405e0232 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:31.193745 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-20e12f79-389d-4b21-8ed1-73da405e0232 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1723 microversion: 2.20 time: 0.157050 Aug 08 02:31:31.194312 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 389/779] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:31:31 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1723 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:31:32.217075 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-181991ab-67c4-407f-8bd3-3cbae132dfe8 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:32.217692 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-181991ab-67c4-407f-8bd3-3cbae132dfe8 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:32.224069 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-181991ab-67c4-407f-8bd3-3cbae132dfe8 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:32.224550 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-181991ab-67c4-407f-8bd3-3cbae132dfe8 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:32.225020 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-181991ab-67c4-407f-8bd3-3cbae132dfe8 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:32.225704 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-181991ab-67c4-407f-8bd3-3cbae132dfe8 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:32.226164 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-181991ab-67c4-407f-8bd3-3cbae132dfe8 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:32.226581 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-181991ab-67c4-407f-8bd3-3cbae132dfe8 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:32.357798 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-181991ab-67c4-407f-8bd3-3cbae132dfe8 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:32.366041 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-181991ab-67c4-407f-8bd3-3cbae132dfe8 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:32.367681 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-181991ab-67c4-407f-8bd3-3cbae132dfe8 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:32.368862 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-181991ab-67c4-407f-8bd3-3cbae132dfe8 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1723 microversion: 2.20 time: 0.155697 Aug 08 02:31:32.369420 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 391/780] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:31:32 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1723 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:31:33.394256 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-34649093-9645-4e6e-a7f7-6a3b563b357a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:33.394921 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-34649093-9645-4e6e-a7f7-6a3b563b357a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:33.401410 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-34649093-9645-4e6e-a7f7-6a3b563b357a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:33.401888 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-34649093-9645-4e6e-a7f7-6a3b563b357a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:33.402456 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-34649093-9645-4e6e-a7f7-6a3b563b357a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:33.404382 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-34649093-9645-4e6e-a7f7-6a3b563b357a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:33.404880 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-34649093-9645-4e6e-a7f7-6a3b563b357a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:33.405306 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-34649093-9645-4e6e-a7f7-6a3b563b357a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:33.538292 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-34649093-9645-4e6e-a7f7-6a3b563b357a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:33.548683 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-34649093-9645-4e6e-a7f7-6a3b563b357a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:33.550409 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-34649093-9645-4e6e-a7f7-6a3b563b357a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:33.551762 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-34649093-9645-4e6e-a7f7-6a3b563b357a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1723 microversion: 2.20 time: 0.161504 Aug 08 02:31:33.552637 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 390/781] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:31:33 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1723 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:31:34.576267 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-465ddb89-92b8-4488-ba87-ee590359bdbf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:34.576873 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-465ddb89-92b8-4488-ba87-ee590359bdbf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:34.585584 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-465ddb89-92b8-4488-ba87-ee590359bdbf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:34.586085 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-465ddb89-92b8-4488-ba87-ee590359bdbf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:34.586523 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-465ddb89-92b8-4488-ba87-ee590359bdbf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:34.588073 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-465ddb89-92b8-4488-ba87-ee590359bdbf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:34.588531 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-465ddb89-92b8-4488-ba87-ee590359bdbf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:34.589097 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-465ddb89-92b8-4488-ba87-ee590359bdbf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:34.716544 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-343a0d0e-aadb-49bd-9495-de661d973561 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:34.718926 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-343a0d0e-aadb-49bd-9495-de661d973561 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: 48b3d4bf-6b20-4327-8f8d-521417405254] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:34.725292 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-343a0d0e-aadb-49bd-9495-de661d973561 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:34.725780 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-343a0d0e-aadb-49bd-9495-de661d973561 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:34.726216 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-343a0d0e-aadb-49bd-9495-de661d973561 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:34.726917 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-343a0d0e-aadb-49bd-9495-de661d973561 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:34.727370 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-343a0d0e-aadb-49bd-9495-de661d973561 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:34.727745 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-465ddb89-92b8-4488-ba87-ee590359bdbf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:34.727901 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-343a0d0e-aadb-49bd-9495-de661d973561 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:34.736401 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-465ddb89-92b8-4488-ba87-ee590359bdbf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:34.737979 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-465ddb89-92b8-4488-ba87-ee590359bdbf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:34.739201 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-465ddb89-92b8-4488-ba87-ee590359bdbf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1744 microversion: 2.20 time: 0.167475 Aug 08 02:31:34.743428 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 392/782] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:31:34 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1744 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:31:34.757994 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-343a0d0e-aadb-49bd-9495-de661d973561 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046157 Aug 08 02:31:34.758551 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 391/783] 10.222.0.27 () {58 vars in 1302 bytes} [Mon Aug 8 02:31:34 2022] GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 08 02:31:34.769671 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-2b45164b-e121-4658-8d4d-1ee1a878d0bb tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:34.770202 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-2b45164b-e121-4658-8d4d-1ee1a878d0bb tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: 48b3d4bf-6b20-4327-8f8d-521417405254] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:34.776154 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2b45164b-e121-4658-8d4d-1ee1a878d0bb tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:34.776643 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2b45164b-e121-4658-8d4d-1ee1a878d0bb tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:34.777115 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2b45164b-e121-4658-8d4d-1ee1a878d0bb tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:34.778480 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2b45164b-e121-4658-8d4d-1ee1a878d0bb tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:34.778974 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2b45164b-e121-4658-8d4d-1ee1a878d0bb tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:34.779406 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2b45164b-e121-4658-8d4d-1ee1a878d0bb tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:34.807270 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-2b45164b-e121-4658-8d4d-1ee1a878d0bb tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254/os-volume_attachments/e89ba9d3-726f-4c26-8f44-1e0b7d17a2b3" status: 200 len: 194 microversion: 2.1 time: 0.041573 Aug 08 02:31:34.807847 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 393/784] 10.222.0.27 () {58 vars in 1413 bytes} [Mon Aug 8 02:31:34 2022] GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254/os-volume_attachments/e89ba9d3-726f-4c26-8f44-1e0b7d17a2b3 => generated 194 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:31:35.764762 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-e45be196-b546-4153-894a-161b08f16c1b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:35.765415 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-e45be196-b546-4153-894a-161b08f16c1b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:35.771844 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e45be196-b546-4153-894a-161b08f16c1b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:35.772331 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e45be196-b546-4153-894a-161b08f16c1b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:35.773089 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e45be196-b546-4153-894a-161b08f16c1b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:35.773515 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e45be196-b546-4153-894a-161b08f16c1b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:35.773971 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e45be196-b546-4153-894a-161b08f16c1b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:35.774579 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e45be196-b546-4153-894a-161b08f16c1b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:35.973017 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-e45be196-b546-4153-894a-161b08f16c1b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:35.981190 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-e45be196-b546-4153-894a-161b08f16c1b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:35.982751 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-e45be196-b546-4153-894a-161b08f16c1b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:35.983929 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-e45be196-b546-4153-894a-161b08f16c1b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1744 microversion: 2.20 time: 0.223219 Aug 08 02:31:35.984490 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 392/785] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:31:35 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1744 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:31:36.482624 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-edfb4f42-1d6b-493c-84ba-32b4e645cbff tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "9b3d4dd3-038e-4e2d-b8f2-833cf39d66b4"}} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:31:36.485124 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-edfb4f42-1d6b-493c-84ba-32b4e645cbff tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: 48b3d4bf-6b20-4327-8f8d-521417405254] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:36.491737 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-edfb4f42-1d6b-493c-84ba-32b4e645cbff tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:36.492245 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-edfb4f42-1d6b-493c-84ba-32b4e645cbff tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:36.492708 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-edfb4f42-1d6b-493c-84ba-32b4e645cbff tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:36.499168 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-edfb4f42-1d6b-493c-84ba-32b4e645cbff tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:36.499168 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-edfb4f42-1d6b-493c-84ba-32b4e645cbff tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:36.499168 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-edfb4f42-1d6b-493c-84ba-32b4e645cbff tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:37.007553 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-3ca11335-3fe5-4e8d-be7a-b12c76c18309 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:37.008108 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-3ca11335-3fe5-4e8d-be7a-b12c76c18309 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:37.015501 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3ca11335-3fe5-4e8d-be7a-b12c76c18309 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:37.015984 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3ca11335-3fe5-4e8d-be7a-b12c76c18309 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:37.016451 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3ca11335-3fe5-4e8d-be7a-b12c76c18309 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:37.017192 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3ca11335-3fe5-4e8d-be7a-b12c76c18309 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:37.017644 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3ca11335-3fe5-4e8d-be7a-b12c76c18309 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:37.018069 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3ca11335-3fe5-4e8d-be7a-b12c76c18309 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:37.229514 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-3ca11335-3fe5-4e8d-be7a-b12c76c18309 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:37.239705 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-3ca11335-3fe5-4e8d-be7a-b12c76c18309 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:37.241953 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-3ca11335-3fe5-4e8d-be7a-b12c76c18309 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:37.254291 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-3ca11335-3fe5-4e8d-be7a-b12c76c18309 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1739 microversion: 2.20 time: 0.248424 Aug 08 02:31:37.255182 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 393/786] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:31:37 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1739 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 08 02:31:37.580034 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-edfb4f42-1d6b-493c-84ba-32b4e645cbff tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "POST /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.101270 Aug 08 02:31:37.588156 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 394/787] 10.222.0.27 () {60 vars in 1323 bytes} [Mon Aug 8 02:31:36 2022] POST /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254/os-volume_attachments => generated 194 bytes in 1111 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 08 02:31:38.278251 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-334e63e3-2608-4779-bedd-c57b6f12da28 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:38.278955 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-334e63e3-2608-4779-bedd-c57b6f12da28 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:38.286290 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-334e63e3-2608-4779-bedd-c57b6f12da28 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:38.287041 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-334e63e3-2608-4779-bedd-c57b6f12da28 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:38.287655 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-334e63e3-2608-4779-bedd-c57b6f12da28 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:38.288622 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-334e63e3-2608-4779-bedd-c57b6f12da28 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:38.289339 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-334e63e3-2608-4779-bedd-c57b6f12da28 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:38.290003 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-334e63e3-2608-4779-bedd-c57b6f12da28 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:38.459745 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [req-2957a67d-b34b-47b8-85ea-23025f93125c req-6c381443-5a97-4c05-ac92-40e070729389 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "946fd571-bd98-44f9-89f4-52f0116b642d", "name": "network-vif-plugged", "status": "completed", "tag": "0bf885ea-ca94-45b2-aafa-7461404e6cf2"}]} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:31:38.468809 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-334e63e3-2608-4779-bedd-c57b6f12da28 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:38.476472 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-2957a67d-b34b-47b8-85ea-23025f93125c req-6c381443-5a97-4c05-ac92-40e070729389 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:38.476995 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-2957a67d-b34b-47b8-85ea-23025f93125c req-6c381443-5a97-4c05-ac92-40e070729389 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:38.477434 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-2957a67d-b34b-47b8-85ea-23025f93125c req-6c381443-5a97-4c05-ac92-40e070729389 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:38.478226 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-334e63e3-2608-4779-bedd-c57b6f12da28 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:38.479872 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-334e63e3-2608-4779-bedd-c57b6f12da28 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:38.481564 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-334e63e3-2608-4779-bedd-c57b6f12da28 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1739 microversion: 2.20 time: 0.207439 Aug 08 02:31:38.482814 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 394/788] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:31:38 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1739 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:31:38.492812 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.compute.server_external_events [req-2957a67d-b34b-47b8-85ea-23025f93125c req-6c381443-5a97-4c05-ac92-40e070729389 service nova] Creating event network-vif-plugged:0bf885ea-ca94-45b2-aafa-7461404e6cf2 for instance 946fd571-bd98-44f9-89f4-52f0116b642d on hv-cinder-74014 Aug 08 02:31:38.498678 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [req-2957a67d-b34b-47b8-85ea-23025f93125c req-6c381443-5a97-4c05-ac92-40e070729389 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042960 Aug 08 02:31:38.499607 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 395/789] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:31:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 08 02:31:39.506288 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-bec7c63f-eeac-46ea-b109-a9b233cbe973 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:39.506852 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-bec7c63f-eeac-46ea-b109-a9b233cbe973 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:39.526500 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-bec7c63f-eeac-46ea-b109-a9b233cbe973 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:39.526500 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-bec7c63f-eeac-46ea-b109-a9b233cbe973 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:39.526500 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-bec7c63f-eeac-46ea-b109-a9b233cbe973 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:39.526500 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-bec7c63f-eeac-46ea-b109-a9b233cbe973 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:39.533947 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-bec7c63f-eeac-46ea-b109-a9b233cbe973 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:39.533947 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-bec7c63f-eeac-46ea-b109-a9b233cbe973 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:39.689783 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-bec7c63f-eeac-46ea-b109-a9b233cbe973 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:39.699962 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-bec7c63f-eeac-46ea-b109-a9b233cbe973 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:39.702673 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-bec7c63f-eeac-46ea-b109-a9b233cbe973 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:39.705559 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-bec7c63f-eeac-46ea-b109-a9b233cbe973 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1739 microversion: 2.20 time: 0.203764 Aug 08 02:31:39.708556 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 395/790] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:31:39 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1739 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:31:40.731937 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-7d5eebd5-bc61-4d0d-bc90-cfd9af9cd0e0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:40.732682 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-7d5eebd5-bc61-4d0d-bc90-cfd9af9cd0e0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:40.743630 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7d5eebd5-bc61-4d0d-bc90-cfd9af9cd0e0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:40.744018 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7d5eebd5-bc61-4d0d-bc90-cfd9af9cd0e0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:40.744448 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7d5eebd5-bc61-4d0d-bc90-cfd9af9cd0e0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:40.745181 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7d5eebd5-bc61-4d0d-bc90-cfd9af9cd0e0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:40.745871 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7d5eebd5-bc61-4d0d-bc90-cfd9af9cd0e0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:40.746538 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7d5eebd5-bc61-4d0d-bc90-cfd9af9cd0e0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:40.955172 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-7d5eebd5-bc61-4d0d-bc90-cfd9af9cd0e0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:40.964130 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-7d5eebd5-bc61-4d0d-bc90-cfd9af9cd0e0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:40.966830 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-7d5eebd5-bc61-4d0d-bc90-cfd9af9cd0e0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:40.967996 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-7d5eebd5-bc61-4d0d-bc90-cfd9af9cd0e0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1739 microversion: 2.20 time: 0.240483 Aug 08 02:31:40.968916 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 396/791] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:31:40 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1739 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:31:41.998328 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-098efcfc-6065-4387-8864-14572ad37e49 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:41.999083 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-098efcfc-6065-4387-8864-14572ad37e49 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:42.006436 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-098efcfc-6065-4387-8864-14572ad37e49 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:42.006932 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-098efcfc-6065-4387-8864-14572ad37e49 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:42.008442 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-098efcfc-6065-4387-8864-14572ad37e49 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:42.008442 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-098efcfc-6065-4387-8864-14572ad37e49 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:42.008624 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-098efcfc-6065-4387-8864-14572ad37e49 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:42.009041 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-098efcfc-6065-4387-8864-14572ad37e49 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:42.172314 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-098efcfc-6065-4387-8864-14572ad37e49 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:42.184307 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-098efcfc-6065-4387-8864-14572ad37e49 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:42.186306 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-098efcfc-6065-4387-8864-14572ad37e49 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:42.187522 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-098efcfc-6065-4387-8864-14572ad37e49 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1739 microversion: 2.20 time: 0.195857 Aug 08 02:31:42.191594 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 396/792] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:31:41 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1739 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:31:43.428343 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-cc0d0d89-bf5a-4ba7-9064-8c7c94c51e7b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:43.428343 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-cc0d0d89-bf5a-4ba7-9064-8c7c94c51e7b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:43.428343 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cc0d0d89-bf5a-4ba7-9064-8c7c94c51e7b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:43.428343 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cc0d0d89-bf5a-4ba7-9064-8c7c94c51e7b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:43.430427 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cc0d0d89-bf5a-4ba7-9064-8c7c94c51e7b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:43.430427 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cc0d0d89-bf5a-4ba7-9064-8c7c94c51e7b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:43.430427 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cc0d0d89-bf5a-4ba7-9064-8c7c94c51e7b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:43.430427 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cc0d0d89-bf5a-4ba7-9064-8c7c94c51e7b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:43.430759 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-cc0d0d89-bf5a-4ba7-9064-8c7c94c51e7b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:43.430759 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-cc0d0d89-bf5a-4ba7-9064-8c7c94c51e7b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:43.430962 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-cc0d0d89-bf5a-4ba7-9064-8c7c94c51e7b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:43.430962 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-cc0d0d89-bf5a-4ba7-9064-8c7c94c51e7b tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1739 microversion: 2.20 time: 0.167355 Aug 08 02:31:43.430962 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 397/793] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:31:43 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1739 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:31:44.451164 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-f9d1f47a-2dbb-4217-a0db-78518b34d95d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:44.454722 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-f9d1f47a-2dbb-4217-a0db-78518b34d95d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:44.464347 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f9d1f47a-2dbb-4217-a0db-78518b34d95d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:44.465500 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f9d1f47a-2dbb-4217-a0db-78518b34d95d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:44.465925 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f9d1f47a-2dbb-4217-a0db-78518b34d95d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:44.466813 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f9d1f47a-2dbb-4217-a0db-78518b34d95d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:44.467422 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f9d1f47a-2dbb-4217-a0db-78518b34d95d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:44.467998 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f9d1f47a-2dbb-4217-a0db-78518b34d95d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:44.635172 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-f9d1f47a-2dbb-4217-a0db-78518b34d95d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:44.647034 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-f9d1f47a-2dbb-4217-a0db-78518b34d95d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:44.648912 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-f9d1f47a-2dbb-4217-a0db-78518b34d95d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:44.650272 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-f9d1f47a-2dbb-4217-a0db-78518b34d95d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1739 microversion: 2.20 time: 0.203799 Aug 08 02:31:44.650824 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 397/794] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:31:44 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1739 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:31:45.395159 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-a66f4134-8621-47b5-9188-d4897dcffea1 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:45.397624 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-a66f4134-8621-47b5-9188-d4897dcffea1 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: 48b3d4bf-6b20-4327-8f8d-521417405254] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:45.400179 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_api:SELECT=38 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:45.408756 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a66f4134-8621-47b5-9188-d4897dcffea1 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:45.409470 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a66f4134-8621-47b5-9188-d4897dcffea1 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:45.410076 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a66f4134-8621-47b5-9188-d4897dcffea1 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:45.410957 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a66f4134-8621-47b5-9188-d4897dcffea1 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:45.411583 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a66f4134-8621-47b5-9188-d4897dcffea1 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:45.412008 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a66f4134-8621-47b5-9188-d4897dcffea1 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:45.452759 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-a66f4134-8621-47b5-9188-d4897dcffea1 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254/os-volume_attachments" status: 200 len: 371 microversion: 2.1 time: 0.061396 Aug 08 02:31:45.454425 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 398/795] 10.222.0.27 () {58 vars in 1302 bytes} [Mon Aug 8 02:31:45 2022] GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254/os-volume_attachments => generated 371 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:31:45.549435 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-7cb06698-1cf3-4ec0-bbc2-ede432446467 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:45.550024 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-7cb06698-1cf3-4ec0-bbc2-ede432446467 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: 48b3d4bf-6b20-4327-8f8d-521417405254] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:45.556696 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7cb06698-1cf3-4ec0-bbc2-ede432446467 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:45.557178 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7cb06698-1cf3-4ec0-bbc2-ede432446467 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:45.557612 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7cb06698-1cf3-4ec0-bbc2-ede432446467 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:45.558296 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7cb06698-1cf3-4ec0-bbc2-ede432446467 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:45.559419 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7cb06698-1cf3-4ec0-bbc2-ede432446467 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:45.560014 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7cb06698-1cf3-4ec0-bbc2-ede432446467 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:45.612096 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-7cb06698-1cf3-4ec0-bbc2-ede432446467 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254/os-volume_attachments/e89ba9d3-726f-4c26-8f44-1e0b7d17a2b3" status: 200 len: 194 microversion: 2.1 time: 0.066767 Aug 08 02:31:45.613393 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 398/796] 10.222.0.27 () {58 vars in 1413 bytes} [Mon Aug 8 02:31:45 2022] GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254/os-volume_attachments/e89ba9d3-726f-4c26-8f44-1e0b7d17a2b3 => generated 194 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:31:45.624740 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-81943e9b-39a1-40b6-b1cc-a72e31a22551 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:45.626956 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-81943e9b-39a1-40b6-b1cc-a72e31a22551 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: 48b3d4bf-6b20-4327-8f8d-521417405254] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:45.631560 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-81943e9b-39a1-40b6-b1cc-a72e31a22551 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:45.632278 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-81943e9b-39a1-40b6-b1cc-a72e31a22551 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:45.632936 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-81943e9b-39a1-40b6-b1cc-a72e31a22551 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:45.633864 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-81943e9b-39a1-40b6-b1cc-a72e31a22551 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:45.634316 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-81943e9b-39a1-40b6-b1cc-a72e31a22551 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:45.634913 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-81943e9b-39a1-40b6-b1cc-a72e31a22551 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:45.677827 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-8e5f024d-64ad-4249-aed7-99ce43c3610f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:45.678749 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-8e5f024d-64ad-4249-aed7-99ce43c3610f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:45.685997 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8e5f024d-64ad-4249-aed7-99ce43c3610f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:45.686489 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8e5f024d-64ad-4249-aed7-99ce43c3610f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:45.686958 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8e5f024d-64ad-4249-aed7-99ce43c3610f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:45.687661 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8e5f024d-64ad-4249-aed7-99ce43c3610f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:45.688118 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8e5f024d-64ad-4249-aed7-99ce43c3610f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:45.688742 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8e5f024d-64ad-4249-aed7-99ce43c3610f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:45.735658 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.objects.instance [None req-81943e9b-39a1-40b6-b1cc-a72e31a22551 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lazy-loading 'services' on Instance uuid 48b3d4bf-6b20-4327-8f8d-521417405254 {{(pid=118352) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 08 02:31:45.861019 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-8e5f024d-64ad-4249-aed7-99ce43c3610f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:45.872197 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-8e5f024d-64ad-4249-aed7-99ce43c3610f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:45.874151 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-8e5f024d-64ad-4249-aed7-99ce43c3610f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:45.875355 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-8e5f024d-64ad-4249-aed7-99ce43c3610f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1739 microversion: 2.20 time: 0.201491 Aug 08 02:31:45.875914 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 399/797] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:31:45 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1739 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 08 02:31:46.019324 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-81943e9b-39a1-40b6-b1cc-a72e31a22551 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "DELETE /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254/os-volume_attachments/e89ba9d3-726f-4c26-8f44-1e0b7d17a2b3" status: 202 len: 0 microversion: 2.1 time: 0.399157 Aug 08 02:31:46.041064 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 399/798] 10.222.0.27 () {58 vars in 1416 bytes} [Mon Aug 8 02:31:45 2022] DELETE /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254/os-volume_attachments/e89ba9d3-726f-4c26-8f44-1e0b7d17a2b3 => generated 0 bytes in 404 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 08 02:31:46.898987 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-a29c07e8-3ba7-4525-8fb0-3fb43d4d7bc0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:46.899765 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-a29c07e8-3ba7-4525-8fb0-3fb43d4d7bc0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:46.906423 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a29c07e8-3ba7-4525-8fb0-3fb43d4d7bc0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:46.907124 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a29c07e8-3ba7-4525-8fb0-3fb43d4d7bc0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:46.907566 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a29c07e8-3ba7-4525-8fb0-3fb43d4d7bc0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:46.908272 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a29c07e8-3ba7-4525-8fb0-3fb43d4d7bc0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:46.908782 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a29c07e8-3ba7-4525-8fb0-3fb43d4d7bc0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:46.909380 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a29c07e8-3ba7-4525-8fb0-3fb43d4d7bc0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:47.069145 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-a29c07e8-3ba7-4525-8fb0-3fb43d4d7bc0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:47.076116 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-a29c07e8-3ba7-4525-8fb0-3fb43d4d7bc0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:47.077742 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-a29c07e8-3ba7-4525-8fb0-3fb43d4d7bc0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:47.078888 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-a29c07e8-3ba7-4525-8fb0-3fb43d4d7bc0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1739 microversion: 2.20 time: 0.183877 Aug 08 02:31:47.079446 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 400/799] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:31:46 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1739 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 08 02:31:48.103488 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-1c1d49b0-7cb7-484d-b875-5fe064a0f1bf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:48.104148 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-1c1d49b0-7cb7-484d-b875-5fe064a0f1bf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:48.110780 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1c1d49b0-7cb7-484d-b875-5fe064a0f1bf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:48.111267 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1c1d49b0-7cb7-484d-b875-5fe064a0f1bf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:48.111740 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1c1d49b0-7cb7-484d-b875-5fe064a0f1bf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:48.112424 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1c1d49b0-7cb7-484d-b875-5fe064a0f1bf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:48.112914 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1c1d49b0-7cb7-484d-b875-5fe064a0f1bf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:48.113344 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1c1d49b0-7cb7-484d-b875-5fe064a0f1bf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:48.224016 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-b62d07fd-3c52-4658-9395-4d9fa82c4243 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:48.224575 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-b62d07fd-3c52-4658-9395-4d9fa82c4243 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: 48b3d4bf-6b20-4327-8f8d-521417405254] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:48.230988 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b62d07fd-3c52-4658-9395-4d9fa82c4243 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:48.231508 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b62d07fd-3c52-4658-9395-4d9fa82c4243 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:48.231945 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b62d07fd-3c52-4658-9395-4d9fa82c4243 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:48.232642 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b62d07fd-3c52-4658-9395-4d9fa82c4243 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:48.233117 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b62d07fd-3c52-4658-9395-4d9fa82c4243 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:48.233537 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b62d07fd-3c52-4658-9395-4d9fa82c4243 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:48.247004 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-1c1d49b0-7cb7-484d-b875-5fe064a0f1bf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:48.255935 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-1c1d49b0-7cb7-484d-b875-5fe064a0f1bf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:48.257464 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-1c1d49b0-7cb7-484d-b875-5fe064a0f1bf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:48.258595 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-1c1d49b0-7cb7-484d-b875-5fe064a0f1bf tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1739 microversion: 2.20 time: 0.160126 Aug 08 02:31:48.259184 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 400/800] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:31:48 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1739 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:31:48.264165 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-b62d07fd-3c52-4658-9395-4d9fa82c4243 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254/os-volume_attachments/9b3d4dd3-038e-4e2d-b8f2-833cf39d66b4" status: 200 len: 194 microversion: 2.1 time: 0.044149 Aug 08 02:31:48.265065 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 401/801] 10.222.0.27 () {58 vars in 1413 bytes} [Mon Aug 8 02:31:48 2022] GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254/os-volume_attachments/9b3d4dd3-038e-4e2d-b8f2-833cf39d66b4 => generated 194 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 08 02:31:48.275242 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-4047fa36-1282-453b-ba7a-360da1c75d73 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:48.275729 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-4047fa36-1282-453b-ba7a-360da1c75d73 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: 48b3d4bf-6b20-4327-8f8d-521417405254] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:48.282482 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4047fa36-1282-453b-ba7a-360da1c75d73 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:48.286231 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4047fa36-1282-453b-ba7a-360da1c75d73 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:48.286231 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4047fa36-1282-453b-ba7a-360da1c75d73 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:48.286231 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4047fa36-1282-453b-ba7a-360da1c75d73 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:48.286231 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4047fa36-1282-453b-ba7a-360da1c75d73 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:48.287507 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4047fa36-1282-453b-ba7a-360da1c75d73 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:48.368563 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.objects.instance [None req-4047fa36-1282-453b-ba7a-360da1c75d73 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lazy-loading 'services' on Instance uuid 48b3d4bf-6b20-4327-8f8d-521417405254 {{(pid=118352) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 08 02:31:48.616831 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-4047fa36-1282-453b-ba7a-360da1c75d73 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "DELETE /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254/os-volume_attachments/9b3d4dd3-038e-4e2d-b8f2-833cf39d66b4" status: 202 len: 0 microversion: 2.1 time: 0.345119 Aug 08 02:31:48.617360 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 401/802] 10.222.0.27 () {58 vars in 1416 bytes} [Mon Aug 8 02:31:48 2022] DELETE /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254/os-volume_attachments/9b3d4dd3-038e-4e2d-b8f2-833cf39d66b4 => generated 0 bytes in 347 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Aug 08 02:31:49.283766 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-e097b0ad-5425-4cbd-9d78-9368f969233e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:49.284477 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-e097b0ad-5425-4cbd-9d78-9368f969233e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:49.291026 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e097b0ad-5425-4cbd-9d78-9368f969233e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:49.291588 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e097b0ad-5425-4cbd-9d78-9368f969233e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:49.291993 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e097b0ad-5425-4cbd-9d78-9368f969233e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:49.292724 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e097b0ad-5425-4cbd-9d78-9368f969233e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:49.293810 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e097b0ad-5425-4cbd-9d78-9368f969233e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:49.294238 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e097b0ad-5425-4cbd-9d78-9368f969233e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:49.441781 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-e097b0ad-5425-4cbd-9d78-9368f969233e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:49.450571 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-e097b0ad-5425-4cbd-9d78-9368f969233e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:49.452247 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-e097b0ad-5425-4cbd-9d78-9368f969233e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:49.453478 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-e097b0ad-5425-4cbd-9d78-9368f969233e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1721 microversion: 2.20 time: 0.173850 Aug 08 02:31:49.454050 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 402/803] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:31:49 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1721 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:31:50.478364 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-bec5f28a-d418-4310-8a00-5218feb0a6b3 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:50.479068 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-bec5f28a-d418-4310-8a00-5218feb0a6b3 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:50.486829 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-bec5f28a-d418-4310-8a00-5218feb0a6b3 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:50.487316 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-bec5f28a-d418-4310-8a00-5218feb0a6b3 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:50.487745 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-bec5f28a-d418-4310-8a00-5218feb0a6b3 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:50.488434 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-bec5f28a-d418-4310-8a00-5218feb0a6b3 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:50.488987 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-bec5f28a-d418-4310-8a00-5218feb0a6b3 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:50.489453 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-bec5f28a-d418-4310-8a00-5218feb0a6b3 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:50.493559 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=2,nova_cell1:INSERT=3 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:50.629633 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-bec5f28a-d418-4310-8a00-5218feb0a6b3 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:50.638114 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-bec5f28a-d418-4310-8a00-5218feb0a6b3 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:50.639840 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-bec5f28a-d418-4310-8a00-5218feb0a6b3 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:50.640975 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-bec5f28a-d418-4310-8a00-5218feb0a6b3 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1721 microversion: 2.20 time: 0.167441 Aug 08 02:31:50.641522 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 402/804] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:31:50 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1721 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:31:50.852033 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-db3139a7-fff8-49ca-abf9-dd29699fc7d9 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:50.854230 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-db3139a7-fff8-49ca-abf9-dd29699fc7d9 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: 48b3d4bf-6b20-4327-8f8d-521417405254] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:50.861544 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-db3139a7-fff8-49ca-abf9-dd29699fc7d9 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:50.862036 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-db3139a7-fff8-49ca-abf9-dd29699fc7d9 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:50.862477 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-db3139a7-fff8-49ca-abf9-dd29699fc7d9 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:50.863204 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-db3139a7-fff8-49ca-abf9-dd29699fc7d9 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:50.863659 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-db3139a7-fff8-49ca-abf9-dd29699fc7d9 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:50.864082 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-db3139a7-fff8-49ca-abf9-dd29699fc7d9 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:50.868152 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:50.893190 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-db3139a7-fff8-49ca-abf9-dd29699fc7d9 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.045137 Aug 08 02:31:50.893846 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 403/805] 10.222.0.27 () {58 vars in 1302 bytes} [Mon Aug 8 02:31:50 2022] GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254/os-volume_attachments => generated 25 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Aug 08 02:31:51.009979 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-2ab09cb3-0179-496b-b140-894ddac994fd tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:51.013571 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-2ab09cb3-0179-496b-b140-894ddac994fd tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: 48b3d4bf-6b20-4327-8f8d-521417405254] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:51.019623 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2ab09cb3-0179-496b-b140-894ddac994fd tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:51.020176 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2ab09cb3-0179-496b-b140-894ddac994fd tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:51.020754 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2ab09cb3-0179-496b-b140-894ddac994fd tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:51.021357 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2ab09cb3-0179-496b-b140-894ddac994fd tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:51.021816 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2ab09cb3-0179-496b-b140-894ddac994fd tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:51.025425 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2ab09cb3-0179-496b-b140-894ddac994fd tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:51.051845 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-2ab09cb3-0179-496b-b140-894ddac994fd tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.045926 Aug 08 02:31:51.052404 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 403/806] 10.222.0.27 () {58 vars in 1302 bytes} [Mon Aug 8 02:31:51 2022] GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254/os-volume_attachments => generated 25 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Aug 08 02:31:51.119362 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-e0efa765-023f-47e8-8d53-b06ef034f106 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:51.119865 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-e0efa765-023f-47e8-8d53-b06ef034f106 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: 48b3d4bf-6b20-4327-8f8d-521417405254] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:51.126899 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e0efa765-023f-47e8-8d53-b06ef034f106 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:51.127443 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e0efa765-023f-47e8-8d53-b06ef034f106 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:51.127880 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e0efa765-023f-47e8-8d53-b06ef034f106 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:51.128580 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e0efa765-023f-47e8-8d53-b06ef034f106 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:51.129119 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e0efa765-023f-47e8-8d53-b06ef034f106 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:51.129637 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e0efa765-023f-47e8-8d53-b06ef034f106 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:51.154890 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-e0efa765-023f-47e8-8d53-b06ef034f106 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: 48b3d4bf-6b20-4327-8f8d-521417405254] Going to try to terminate instance {{(pid=118351) delete /opt/stack/nova/nova/compute/api.py:2666}} Aug 08 02:31:51.523005 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-e0efa765-023f-47e8-8d53-b06ef034f106 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "DELETE /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254" status: 204 len: 0 microversion: 2.1 time: 0.407750 Aug 08 02:31:51.523601 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 404/807] 10.222.0.27 () {58 vars in 1239 bytes} [Mon Aug 8 02:31:51 2022] DELETE /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 08 02:31:51.533586 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-7b110261-8338-406a-a619-d00558c297f5 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:51.534250 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-7b110261-8338-406a-a619-d00558c297f5 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: 48b3d4bf-6b20-4327-8f8d-521417405254] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:51.541769 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7b110261-8338-406a-a619-d00558c297f5 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:51.542337 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7b110261-8338-406a-a619-d00558c297f5 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:51.543473 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7b110261-8338-406a-a619-d00558c297f5 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:51.544220 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7b110261-8338-406a-a619-d00558c297f5 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:51.545635 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7b110261-8338-406a-a619-d00558c297f5 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:51.546209 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7b110261-8338-406a-a619-d00558c297f5 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:51.664019 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-db3aaf12-4e1d-455c-b6c2-a34c19ea2748 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:51.664739 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-db3aaf12-4e1d-455c-b6c2-a34c19ea2748 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:51.671928 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-db3aaf12-4e1d-455c-b6c2-a34c19ea2748 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:51.672423 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-db3aaf12-4e1d-455c-b6c2-a34c19ea2748 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:51.672896 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-db3aaf12-4e1d-455c-b6c2-a34c19ea2748 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:51.673593 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-db3aaf12-4e1d-455c-b6c2-a34c19ea2748 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:51.674047 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-db3aaf12-4e1d-455c-b6c2-a34c19ea2748 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:51.674469 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-db3aaf12-4e1d-455c-b6c2-a34c19ea2748 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:51.686195 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-7b110261-8338-406a-a619-d00558c297f5 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:51.693960 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-7b110261-8338-406a-a619-d00558c297f5 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254" status: 200 len: 1674 microversion: 2.1 time: 0.165330 Aug 08 02:31:51.694529 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 404/808] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:31:51 2022] GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254 => generated 1674 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:31:51.805208 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-db3aaf12-4e1d-455c-b6c2-a34c19ea2748 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:51.814201 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-db3aaf12-4e1d-455c-b6c2-a34c19ea2748 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:51.819701 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-db3aaf12-4e1d-455c-b6c2-a34c19ea2748 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:51.819701 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-db3aaf12-4e1d-455c-b6c2-a34c19ea2748 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1721 microversion: 2.20 time: 0.157203 Aug 08 02:31:51.819701 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 405/809] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:31:51 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1721 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 08 02:31:52.370641 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [req-4d6f65db-e3ef-4c99-b58c-c70129e5da4e req-efe23716-8bb7-46a6-83b3-255b82bcd017 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "946fd571-bd98-44f9-89f4-52f0116b642d", "name": "network-vif-unplugged", "status": "completed", "tag": "0bf885ea-ca94-45b2-aafa-7461404e6cf2"}]} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:31:52.380439 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-4d6f65db-e3ef-4c99-b58c-c70129e5da4e req-efe23716-8bb7-46a6-83b3-255b82bcd017 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:52.380956 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-4d6f65db-e3ef-4c99-b58c-c70129e5da4e req-efe23716-8bb7-46a6-83b3-255b82bcd017 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:52.381390 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-4d6f65db-e3ef-4c99-b58c-c70129e5da4e req-efe23716-8bb7-46a6-83b3-255b82bcd017 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:52.392502 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.compute.server_external_events [req-4d6f65db-e3ef-4c99-b58c-c70129e5da4e req-efe23716-8bb7-46a6-83b3-255b82bcd017 service nova] Creating event network-vif-unplugged:0bf885ea-ca94-45b2-aafa-7461404e6cf2 for instance 946fd571-bd98-44f9-89f4-52f0116b642d on hv-cinder-74014 Aug 08 02:31:52.397315 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [req-4d6f65db-e3ef-4c99-b58c-c70129e5da4e req-efe23716-8bb7-46a6-83b3-255b82bcd017 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.030425 Aug 08 02:31:52.397885 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 405/810] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:31:52 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:31:52.716918 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-e4cd36d8-c1d3-4075-be6b-c1b8453e6ab6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:52.717492 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-e4cd36d8-c1d3-4075-be6b-c1b8453e6ab6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: 48b3d4bf-6b20-4327-8f8d-521417405254] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:52.720104 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_api:SELECT=49,nova_api:UPDATE=1 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:52.725125 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e4cd36d8-c1d3-4075-be6b-c1b8453e6ab6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:52.730037 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e4cd36d8-c1d3-4075-be6b-c1b8453e6ab6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:52.730037 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e4cd36d8-c1d3-4075-be6b-c1b8453e6ab6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:52.730037 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e4cd36d8-c1d3-4075-be6b-c1b8453e6ab6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:52.730037 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e4cd36d8-c1d3-4075-be6b-c1b8453e6ab6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:52.730415 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e4cd36d8-c1d3-4075-be6b-c1b8453e6ab6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:52.842689 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-949a7f6b-6298-4495-b541-721a7ac46d76 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:52.843379 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-949a7f6b-6298-4495-b541-721a7ac46d76 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:52.849539 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-949a7f6b-6298-4495-b541-721a7ac46d76 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:52.850059 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-949a7f6b-6298-4495-b541-721a7ac46d76 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:52.850638 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-949a7f6b-6298-4495-b541-721a7ac46d76 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:52.851355 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-949a7f6b-6298-4495-b541-721a7ac46d76 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:52.851804 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-949a7f6b-6298-4495-b541-721a7ac46d76 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:52.852231 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-949a7f6b-6298-4495-b541-721a7ac46d76 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:52.875847 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-e4cd36d8-c1d3-4075-be6b-c1b8453e6ab6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:52.883465 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-e4cd36d8-c1d3-4075-be6b-c1b8453e6ab6 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254" status: 200 len: 1674 microversion: 2.1 time: 0.171729 Aug 08 02:31:52.884041 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 406/811] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:31:52 2022] GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254 => generated 1674 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:31:53.022405 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-949a7f6b-6298-4495-b541-721a7ac46d76 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:53.034914 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-949a7f6b-6298-4495-b541-721a7ac46d76 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:53.036445 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-949a7f6b-6298-4495-b541-721a7ac46d76 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:53.037636 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-949a7f6b-6298-4495-b541-721a7ac46d76 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1721 microversion: 2.20 time: 0.200652 Aug 08 02:31:53.038220 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 406/812] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:31:52 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1721 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 08 02:31:53.905709 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-aaf4c78e-f160-4b18-82da-c6951bac1537 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:53.907263 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-aaf4c78e-f160-4b18-82da-c6951bac1537 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: 48b3d4bf-6b20-4327-8f8d-521417405254] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:53.914469 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-aaf4c78e-f160-4b18-82da-c6951bac1537 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:53.915393 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-aaf4c78e-f160-4b18-82da-c6951bac1537 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:53.916024 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-aaf4c78e-f160-4b18-82da-c6951bac1537 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:53.917689 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-aaf4c78e-f160-4b18-82da-c6951bac1537 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:53.918982 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-aaf4c78e-f160-4b18-82da-c6951bac1537 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:53.919719 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-aaf4c78e-f160-4b18-82da-c6951bac1537 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:54.062807 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-090883fa-cc3f-4e77-b1bc-3a0045203585 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:54.064025 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-090883fa-cc3f-4e77-b1bc-3a0045203585 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:54.071340 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-090883fa-cc3f-4e77-b1bc-3a0045203585 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:54.072110 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-090883fa-cc3f-4e77-b1bc-3a0045203585 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:54.072496 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-090883fa-cc3f-4e77-b1bc-3a0045203585 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:54.073283 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-090883fa-cc3f-4e77-b1bc-3a0045203585 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:54.073838 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-090883fa-cc3f-4e77-b1bc-3a0045203585 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:54.075100 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-090883fa-cc3f-4e77-b1bc-3a0045203585 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:54.114000 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-aaf4c78e-f160-4b18-82da-c6951bac1537 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:54.122656 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-aaf4c78e-f160-4b18-82da-c6951bac1537 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254" status: 200 len: 1674 microversion: 2.1 time: 0.221036 Aug 08 02:31:54.123242 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 407/813] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:31:53 2022] GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254 => generated 1674 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:31:54.213903 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-090883fa-cc3f-4e77-b1bc-3a0045203585 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:54.222195 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-090883fa-cc3f-4e77-b1bc-3a0045203585 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:54.223714 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-090883fa-cc3f-4e77-b1bc-3a0045203585 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:54.224898 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-090883fa-cc3f-4e77-b1bc-3a0045203585 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1664 microversion: 2.20 time: 0.167277 Aug 08 02:31:54.225454 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 407/814] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:31:54 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1664 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 08 02:31:54.248083 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-d955c15c-54c4-4944-861a-f5efce4131e7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "adcc1630-f29f-44ec-9be8-a5493405ff12"}} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:31:54.250493 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-d955c15c-54c4-4944-861a-f5efce4131e7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:54.256607 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d955c15c-54c4-4944-861a-f5efce4131e7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:54.257128 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d955c15c-54c4-4944-861a-f5efce4131e7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:54.257571 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d955c15c-54c4-4944-861a-f5efce4131e7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:54.258290 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d955c15c-54c4-4944-861a-f5efce4131e7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:54.258744 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d955c15c-54c4-4944-861a-f5efce4131e7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:54.259199 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d955c15c-54c4-4944-861a-f5efce4131e7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:54.672347 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [req-244e5659-5ad5-4da1-8fe3-d6024825d9ea req-acc8f63e-2e80-4887-b78c-1569343a8bb2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "48b3d4bf-6b20-4327-8f8d-521417405254", "name": "network-vif-deleted", "tag": "961375ea-c683-4138-905d-e3bf380590df"}]} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:31:54.683296 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-244e5659-5ad5-4da1-8fe3-d6024825d9ea req-acc8f63e-2e80-4887-b78c-1569343a8bb2 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:54.686890 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-244e5659-5ad5-4da1-8fe3-d6024825d9ea req-acc8f63e-2e80-4887-b78c-1569343a8bb2 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:54.686890 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-244e5659-5ad5-4da1-8fe3-d6024825d9ea req-acc8f63e-2e80-4887-b78c-1569343a8bb2 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:54.699058 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.compute.server_external_events [req-244e5659-5ad5-4da1-8fe3-d6024825d9ea req-acc8f63e-2e80-4887-b78c-1569343a8bb2 service nova] Creating event network-vif-deleted:961375ea-c683-4138-905d-e3bf380590df for instance 48b3d4bf-6b20-4327-8f8d-521417405254 on hv-cinder-74014 Aug 08 02:31:54.709660 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [req-244e5659-5ad5-4da1-8fe3-d6024825d9ea req-acc8f63e-2e80-4887-b78c-1569343a8bb2 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037704 Aug 08 02:31:54.709660 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 408/815] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:31:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:31:55.147720 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-729ca087-9c18-45a6-aba0-eb6b0951ad6b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:55.149619 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-729ca087-9c18-45a6-aba0-eb6b0951ad6b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: 48b3d4bf-6b20-4327-8f8d-521417405254] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:55.159542 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-729ca087-9c18-45a6-aba0-eb6b0951ad6b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:55.160251 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-729ca087-9c18-45a6-aba0-eb6b0951ad6b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:55.160891 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-729ca087-9c18-45a6-aba0-eb6b0951ad6b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:55.161596 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-729ca087-9c18-45a6-aba0-eb6b0951ad6b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:55.162087 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-729ca087-9c18-45a6-aba0-eb6b0951ad6b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:55.162463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-729ca087-9c18-45a6-aba0-eb6b0951ad6b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:55.204170 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-729ca087-9c18-45a6-aba0-eb6b0951ad6b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:55.204904 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-729ca087-9c18-45a6-aba0-eb6b0951ad6b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:55.205368 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-729ca087-9c18-45a6-aba0-eb6b0951ad6b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:55.205816 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.objects.instance [None req-729ca087-9c18-45a6-aba0-eb6b0951ad6b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lazy-loading 'fault' on Instance uuid 48b3d4bf-6b20-4327-8f8d-521417405254 {{(pid=118352) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 08 02:31:55.252285 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-729ca087-9c18-45a6-aba0-eb6b0951ad6b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eb4c92edcb024e32a549650eb85f6952', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee8baa7d6e294e27ae2fb12eded3709d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:55.259864 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-729ca087-9c18-45a6-aba0-eb6b0951ad6b tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254" status: 200 len: 1328 microversion: 2.1 time: 0.118137 Aug 08 02:31:55.260431 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 409/816] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:31:55 2022] GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254 => generated 1328 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:31:55.497350 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-d955c15c-54c4-4944-861a-f5efce4131e7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "POST /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 1.253810 Aug 08 02:31:55.497974 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 408/817] 10.222.0.27 () {62 vars in 1364 bytes} [Mon Aug 8 02:31:54 2022] POST /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d/os-volume_attachments => generated 188 bytes in 1255 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 08 02:31:55.630154 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-504f61fe-d817-4f47-b8ee-f1495208de33 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:55.630154 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-504f61fe-d817-4f47-b8ee-f1495208de33 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: cc2b42f1-3dd0-4090-a905-93283047aea3] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:55.630154 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-504f61fe-d817-4f47-b8ee-f1495208de33 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:55.630154 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-504f61fe-d817-4f47-b8ee-f1495208de33 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:55.630541 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-504f61fe-d817-4f47-b8ee-f1495208de33 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:55.630541 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-504f61fe-d817-4f47-b8ee-f1495208de33 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:55.630541 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-504f61fe-d817-4f47-b8ee-f1495208de33 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:55.630541 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-504f61fe-d817-4f47-b8ee-f1495208de33 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:55.641467 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-811e1c6a-9a3d-4c90-90bc-21bff9237a69 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:55.641982 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-811e1c6a-9a3d-4c90-90bc-21bff9237a69 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:55.650355 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-811e1c6a-9a3d-4c90-90bc-21bff9237a69 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:55.660395 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-811e1c6a-9a3d-4c90-90bc-21bff9237a69 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:55.660395 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-811e1c6a-9a3d-4c90-90bc-21bff9237a69 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:55.660395 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-811e1c6a-9a3d-4c90-90bc-21bff9237a69 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:55.660395 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-811e1c6a-9a3d-4c90-90bc-21bff9237a69 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:55.660871 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-811e1c6a-9a3d-4c90-90bc-21bff9237a69 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:55.666432 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-504f61fe-d817-4f47-b8ee-f1495208de33 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3/ips" status: 200 len: 151 microversion: 2.1 time: 0.051704 Aug 08 02:31:55.667333 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 410/818] 10.222.0.27 () {58 vars in 1248 bytes} [Mon Aug 8 02:31:55 2022] GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3/ips => generated 151 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:31:56.283603 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-0981eb0d-a1c2-4f09-9e27-21a6e37edd68 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:56.284388 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-0981eb0d-a1c2-4f09-9e27-21a6e37edd68 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: 48b3d4bf-6b20-4327-8f8d-521417405254] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:56.291451 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0981eb0d-a1c2-4f09-9e27-21a6e37edd68 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:56.292105 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0981eb0d-a1c2-4f09-9e27-21a6e37edd68 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:56.292644 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0981eb0d-a1c2-4f09-9e27-21a6e37edd68 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:56.294339 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0981eb0d-a1c2-4f09-9e27-21a6e37edd68 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:56.294928 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0981eb0d-a1c2-4f09-9e27-21a6e37edd68 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:56.295377 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0981eb0d-a1c2-4f09-9e27-21a6e37edd68 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:56.315197 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.wsgi [None req-0981eb0d-a1c2-4f09-9e27-21a6e37edd68 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] HTTP exception thrown: Instance 48b3d4bf-6b20-4327-8f8d-521417405254 could not be found. Aug 08 02:31:56.315197 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-0981eb0d-a1c2-4f09-9e27-21a6e37edd68 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Request method failure captured: Aug 08 02:31:56.315197 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: request: GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254 HTTP/1.1 Aug 08 02:31:56.315197 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept: application/json Aug 08 02:31:56.315197 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept-Encoding: identity Aug 08 02:31:56.315197 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Connection: close Aug 08 02:31:56.315197 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Length: 0 Aug 08 02:31:56.315197 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Type: application/json Aug 08 02:31:56.315197 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Host: 10.222.0.27 Aug 08 02:31:56.315197 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Openstack-System-Scope: None Aug 08 02:31:56.315197 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: User-Agent: python-urllib3/1.26.11 Aug 08 02:31:56.315197 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Auth-Token: gAAAAABi8HQalNSG8U0tawdT8BYmZ3dO_Z_XJ7b_zGy3jhslHecNN8lPHCadrD9L1wE_fNDeX2KYU182-SQacEPppcF5Uwunh8fURGrax-nwHUph2qRhEJoRudpg1Feo8sNMetQ07fI55mt7EEM91lToca7t2N_CdvJ1rSwFvtzAdQ4uUEvan-8 Aug 08 02:31:56.315197 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Id: None Aug 08 02:31:56.315197 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Name: None Aug 08 02:31:56.315197 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Identity-Status: Confirmed Aug 08 02:31:56.315197 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Is-Admin-Project: True Aug 08 02:31:56.315197 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Id: default Aug 08 02:31:56.315197 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Name: Default Aug 08 02:31:56.315197 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Id: ee8baa7d6e294e27ae2fb12eded3709d Aug 08 02:31:56.315197 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Name: tempest-AttachVolumeTestJSON-827912071 Aug 08 02:31:56.315197 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Role: reader,member Aug 08 02:31:56.315197 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Roles: reader,member Aug 08 02:31:56.316977 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/ee8baa7d6e294e27ae2fb12eded3709d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/ee8baa7d6e294e27ae2fb12eded3709d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_ee8baa7d6e294e27ae2fb12eded3709d", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/ee8baa7d6e294e27ae2fb12eded3709d"}]}] Aug 08 02:31:56.316977 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant: tempest-AttachVolumeTestJSON-827912071 Aug 08 02:31:56.316977 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Id: ee8baa7d6e294e27ae2fb12eded3709d Aug 08 02:31:56.316977 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Name: tempest-AttachVolumeTestJSON-827912071 Aug 08 02:31:56.316977 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User: tempest-AttachVolumeTestJSON-827912071-project Aug 08 02:31:56.316977 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Id: default Aug 08 02:31:56.316977 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Name: Default Aug 08 02:31:56.316977 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Id: eb4c92edcb024e32a549650eb85f6952 Aug 08 02:31:56.316977 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Name: tempest-AttachVolumeTestJSON-827912071-project Aug 08 02:31:56.316977 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: method: > Aug 08 02:31:56.316977 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: exception: Instance 48b3d4bf-6b20-4327-8f8d-521417405254 could not be found. Aug 08 02:31:56.316977 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: action_args: {'id': '48b3d4bf-6b20-4327-8f8d-521417405254'} Aug 08 02:31:56.317869 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:31:56.317869 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:31:56.317869 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 08 02:31:56.317869 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 08 02:31:56.317869 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 08 02:31:56.317869 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 08 02:31:56.317869 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 08 02:31:56.317869 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 08 02:31:56.317869 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 08 02:31:56.317869 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 08 02:31:56.317869 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 48b3d4bf-6b20-4327-8f8d-521417405254 could not be found. Aug 08 02:31:56.317869 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:31:56.317869 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:31:56.317869 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:31:56.317869 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:31:56.317869 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:31:56.317869 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:31:56.317869 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:31:56.317869 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:31:56.317869 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:31:56.319414 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:31:56.319414 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 08 02:31:56.319414 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 08 02:31:56.319414 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 08 02:31:56.319414 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 08 02:31:56.319414 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 08 02:31:56.319414 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:31:56.319414 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 48b3d4bf-6b20-4327-8f8d-521417405254 could not be found. Aug 08 02:31:56.319414 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:31:56.319414 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:31:56.319414 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:31:56.319414 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:31:56.319414 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:31:56.319414 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:31:56.319414 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:31:56.319414 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:31:56.319414 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 48b3d4bf-6b20-4327-8f8d-521417405254 could not be found. Aug 08 02:31:56.319414 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:31:56.321169 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-0981eb0d-a1c2-4f09-9e27-21a6e37edd68 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Returning 404 to user: Instance 48b3d4bf-6b20-4327-8f8d-521417405254 could not be found. {{(pid=118352) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:31:56.321169 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-0981eb0d-a1c2-4f09-9e27-21a6e37edd68 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254" status: 404 len: 111 microversion: 2.1 time: 0.033787 Aug 08 02:31:56.321169 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 411/819] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:31:56 2022] GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Aug 08 02:31:56.322388 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-c3c00a23-12e1-47e3-9baa-65743b19cda7 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '.version_select at 0x7efc7405c940>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:56.666739 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-c3c00a23-12e1-47e3-9baa-65743b19cda7 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-2085281366" status: 202 len: 0 microversion: 2.1 time: 0.348087 Aug 08 02:31:56.667401 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 412/820] 10.222.0.27 () {58 vars in 1221 bytes} [Mon Aug 8 02:31:56 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-2085281366 => generated 0 bytes in 349 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Aug 08 02:31:57.339400 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-811e1c6a-9a3d-4c90-90bc-21bff9237a69 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "DELETE /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d/os-volume_attachments/adcc1630-f29f-44ec-9be8-a5493405ff12" status: 202 len: 0 microversion: 2.20 time: 1.703389 Aug 08 02:31:57.340801 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 409/821] 10.222.0.27 () {60 vars in 1457 bytes} [Mon Aug 8 02:31:55 2022] DELETE /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d/os-volume_attachments/adcc1630-f29f-44ec-9be8-a5493405ff12 => generated 0 bytes in 1705 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Aug 08 02:31:57.403397 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-cb0bfe78-9289-4728-bb24-c85ba6adb5e6 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:31:57.404002 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-cb0bfe78-9289-4728-bb24-c85ba6adb5e6 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:57.410836 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cb0bfe78-9289-4728-bb24-c85ba6adb5e6 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:57.411326 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cb0bfe78-9289-4728-bb24-c85ba6adb5e6 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:57.411763 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cb0bfe78-9289-4728-bb24-c85ba6adb5e6 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:57.412454 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cb0bfe78-9289-4728-bb24-c85ba6adb5e6 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:57.412954 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cb0bfe78-9289-4728-bb24-c85ba6adb5e6 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:57.413381 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cb0bfe78-9289-4728-bb24-c85ba6adb5e6 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:57.457905 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-cb0bfe78-9289-4728-bb24-c85ba6adb5e6 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=118352) unshelve /opt/stack/nova/nova/compute/api.py:4531}} Aug 08 02:31:57.539660 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-a4a387ea-56ab-4c07-9493-e5537ca54439 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "872d9b80-e690-4de4-8cff-d1e23d8de6cf"}} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:31:57.543396 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-a4a387ea-56ab-4c07-9493-e5537ca54439 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: cc2b42f1-3dd0-4090-a905-93283047aea3] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:57.550165 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a4a387ea-56ab-4c07-9493-e5537ca54439 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:57.550165 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a4a387ea-56ab-4c07-9493-e5537ca54439 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:57.550481 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a4a387ea-56ab-4c07-9493-e5537ca54439 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:57.551069 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a4a387ea-56ab-4c07-9493-e5537ca54439 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:57.551524 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a4a387ea-56ab-4c07-9493-e5537ca54439 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:57.551950 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a4a387ea-56ab-4c07-9493-e5537ca54439 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:57.819556 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-cb0bfe78-9289-4728-bb24-c85ba6adb5e6 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "POST /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d/action" status: 202 len: 0 microversion: 2.20 time: 0.421372 Aug 08 02:31:57.820480 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 413/822] 10.222.0.27 () {62 vars in 1319 bytes} [Mon Aug 8 02:31:57 2022] POST /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d/action => generated 0 bytes in 423 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Aug 08 02:31:57.830608 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-1c8ec803-2159-424b-8a85-625c7167f2c4 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:57.831230 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-1c8ec803-2159-424b-8a85-625c7167f2c4 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:57.837770 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1c8ec803-2159-424b-8a85-625c7167f2c4 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:57.837770 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1c8ec803-2159-424b-8a85-625c7167f2c4 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:57.838219 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1c8ec803-2159-424b-8a85-625c7167f2c4 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:57.838922 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1c8ec803-2159-424b-8a85-625c7167f2c4 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:57.839380 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1c8ec803-2159-424b-8a85-625c7167f2c4 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:57.839945 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1c8ec803-2159-424b-8a85-625c7167f2c4 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:58.004614 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-1c8ec803-2159-424b-8a85-625c7167f2c4 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:58.013120 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-1c8ec803-2159-424b-8a85-625c7167f2c4 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:58.014961 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-1c8ec803-2159-424b-8a85-625c7167f2c4 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:58.016637 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-1c8ec803-2159-424b-8a85-625c7167f2c4 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1672 microversion: 2.20 time: 0.191093 Aug 08 02:31:58.028227 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 414/823] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:31:57 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1672 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 08 02:31:58.731260 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-a4a387ea-56ab-4c07-9493-e5537ca54439 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "POST /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.194657 Aug 08 02:31:58.732971 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 410/824] 10.222.0.27 () {60 vars in 1323 bytes} [Mon Aug 8 02:31:57 2022] POST /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3/os-volume_attachments => generated 194 bytes in 1198 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 08 02:31:59.039137 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-0adf82b6-694f-45d4-b8e4-491ba7c7ffa9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:31:59.039733 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-0adf82b6-694f-45d4-b8e4-491ba7c7ffa9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:31:59.046712 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0adf82b6-694f-45d4-b8e4-491ba7c7ffa9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:59.047206 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0adf82b6-694f-45d4-b8e4-491ba7c7ffa9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:59.047651 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0adf82b6-694f-45d4-b8e4-491ba7c7ffa9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:59.048348 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0adf82b6-694f-45d4-b8e4-491ba7c7ffa9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:31:59.048840 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0adf82b6-694f-45d4-b8e4-491ba7c7ffa9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:31:59.049278 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0adf82b6-694f-45d4-b8e4-491ba7c7ffa9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:31:59.180393 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-0adf82b6-694f-45d4-b8e4-491ba7c7ffa9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:59.188526 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-0adf82b6-694f-45d4-b8e4-491ba7c7ffa9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:59.190052 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-0adf82b6-694f-45d4-b8e4-491ba7c7ffa9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:31:59.191195 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-0adf82b6-694f-45d4-b8e4-491ba7c7ffa9 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1672 microversion: 2.20 time: 0.155895 Aug 08 02:31:59.191749 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 415/825] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:31:59 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1672 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:32:00.215926 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-800afd37-2896-4086-b587-637bbd8c5c8e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:00.216777 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-800afd37-2896-4086-b587-637bbd8c5c8e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:00.223649 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-800afd37-2896-4086-b587-637bbd8c5c8e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:00.224146 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-800afd37-2896-4086-b587-637bbd8c5c8e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:00.224580 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-800afd37-2896-4086-b587-637bbd8c5c8e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:00.225403 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-800afd37-2896-4086-b587-637bbd8c5c8e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:00.225835 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-800afd37-2896-4086-b587-637bbd8c5c8e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:00.226258 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-800afd37-2896-4086-b587-637bbd8c5c8e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:00.360019 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-62d88710-e2c1-405a-917f-da2fe81cc930 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:00.360644 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-62d88710-e2c1-405a-917f-da2fe81cc930 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: 48b3d4bf-6b20-4327-8f8d-521417405254] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:00.367315 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-62d88710-e2c1-405a-917f-da2fe81cc930 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:00.367696 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-62d88710-e2c1-405a-917f-da2fe81cc930 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:00.368113 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-800afd37-2896-4086-b587-637bbd8c5c8e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:00.368244 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-62d88710-e2c1-405a-917f-da2fe81cc930 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:00.368854 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-62d88710-e2c1-405a-917f-da2fe81cc930 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:00.369387 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-62d88710-e2c1-405a-917f-da2fe81cc930 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:00.369734 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-62d88710-e2c1-405a-917f-da2fe81cc930 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:00.370763 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell1:SELECT=82,nova_cell1:UPDATE=11,nova_cell1:INSERT=7 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:00.378400 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-800afd37-2896-4086-b587-637bbd8c5c8e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:00.380116 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-800afd37-2896-4086-b587-637bbd8c5c8e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:00.381317 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-800afd37-2896-4086-b587-637bbd8c5c8e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1730 microversion: 2.20 time: 0.170271 Aug 08 02:32:00.381883 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 411/826] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:32:00 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1730 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:32:00.391805 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.wsgi [None req-62d88710-e2c1-405a-917f-da2fe81cc930 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] HTTP exception thrown: Instance 48b3d4bf-6b20-4327-8f8d-521417405254 could not be found. Aug 08 02:32:00.393716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-62d88710-e2c1-405a-917f-da2fe81cc930 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Request method failure captured: Aug 08 02:32:00.393716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: request: DELETE /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254 HTTP/1.1 Aug 08 02:32:00.393716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept: application/json Aug 08 02:32:00.393716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept-Encoding: identity Aug 08 02:32:00.393716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Connection: close Aug 08 02:32:00.393716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Length: 0 Aug 08 02:32:00.393716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Type: application/json Aug 08 02:32:00.393716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Host: 10.222.0.27 Aug 08 02:32:00.393716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Openstack-System-Scope: None Aug 08 02:32:00.393716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: User-Agent: python-urllib3/1.26.11 Aug 08 02:32:00.393716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Auth-Token: gAAAAABi8HQalNSG8U0tawdT8BYmZ3dO_Z_XJ7b_zGy3jhslHecNN8lPHCadrD9L1wE_fNDeX2KYU182-SQacEPppcF5Uwunh8fURGrax-nwHUph2qRhEJoRudpg1Feo8sNMetQ07fI55mt7EEM91lToca7t2N_CdvJ1rSwFvtzAdQ4uUEvan-8 Aug 08 02:32:00.393716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Id: None Aug 08 02:32:00.393716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Name: None Aug 08 02:32:00.393716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Identity-Status: Confirmed Aug 08 02:32:00.393716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Is-Admin-Project: True Aug 08 02:32:00.393716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Id: default Aug 08 02:32:00.393716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Name: Default Aug 08 02:32:00.393716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Id: ee8baa7d6e294e27ae2fb12eded3709d Aug 08 02:32:00.393716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Name: tempest-AttachVolumeTestJSON-827912071 Aug 08 02:32:00.393716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Role: reader,member Aug 08 02:32:00.393716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Roles: reader,member Aug 08 02:32:00.395247 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/ee8baa7d6e294e27ae2fb12eded3709d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/ee8baa7d6e294e27ae2fb12eded3709d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_ee8baa7d6e294e27ae2fb12eded3709d", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/ee8baa7d6e294e27ae2fb12eded3709d"}]}] Aug 08 02:32:00.395247 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant: tempest-AttachVolumeTestJSON-827912071 Aug 08 02:32:00.395247 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Id: ee8baa7d6e294e27ae2fb12eded3709d Aug 08 02:32:00.395247 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Name: tempest-AttachVolumeTestJSON-827912071 Aug 08 02:32:00.395247 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User: tempest-AttachVolumeTestJSON-827912071-project Aug 08 02:32:00.395247 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Id: default Aug 08 02:32:00.395247 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Name: Default Aug 08 02:32:00.395247 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Id: eb4c92edcb024e32a549650eb85f6952 Aug 08 02:32:00.395247 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Name: tempest-AttachVolumeTestJSON-827912071-project Aug 08 02:32:00.395247 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: method: > Aug 08 02:32:00.395247 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: exception: Instance 48b3d4bf-6b20-4327-8f8d-521417405254 could not be found. Aug 08 02:32:00.395247 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: action_args: {'id': '48b3d4bf-6b20-4327-8f8d-521417405254'} Aug 08 02:32:00.396119 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:32:00.396119 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:00.396119 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 08 02:32:00.396119 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 08 02:32:00.396119 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 08 02:32:00.396119 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 08 02:32:00.396119 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 08 02:32:00.396119 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 08 02:32:00.396119 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 08 02:32:00.396119 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 08 02:32:00.396119 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 48b3d4bf-6b20-4327-8f8d-521417405254 could not be found. Aug 08 02:32:00.396119 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:00.396119 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:32:00.396119 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:00.396119 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:00.396119 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:32:00.396119 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:32:00.396119 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:32:00.396119 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:32:00.396119 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:32:00.397547 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:32:00.397547 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 08 02:32:00.397547 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 08 02:32:00.397547 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 08 02:32:00.397547 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 08 02:32:00.397547 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 08 02:32:00.397547 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 08 02:32:00.397547 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 08 02:32:00.397547 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:32:00.397547 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 48b3d4bf-6b20-4327-8f8d-521417405254 could not be found. Aug 08 02:32:00.397547 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:00.397547 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:32:00.397547 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:00.397547 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:00.397547 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:32:00.397547 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:32:00.397547 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:32:00.397547 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:32:00.397547 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 48b3d4bf-6b20-4327-8f8d-521417405254 could not be found. Aug 08 02:32:00.397547 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:00.400040 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-62d88710-e2c1-405a-917f-da2fe81cc930 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Returning 404 to user: Instance 48b3d4bf-6b20-4327-8f8d-521417405254 could not be found. {{(pid=118352) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:32:00.400916 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-62d88710-e2c1-405a-917f-da2fe81cc930 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "DELETE /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254" status: 404 len: 111 microversion: 2.1 time: 0.045501 Aug 08 02:32:00.401474 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 416/827] 10.222.0.27 () {58 vars in 1239 bytes} [Mon Aug 8 02:32:00 2022] DELETE /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254 => generated 111 bytes in 47 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Aug 08 02:32:00.409909 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-ede7cd4d-6818-46c8-aa8c-01a059306078 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:00.410714 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-ede7cd4d-6818-46c8-aa8c-01a059306078 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: 48b3d4bf-6b20-4327-8f8d-521417405254] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:00.417438 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ede7cd4d-6818-46c8-aa8c-01a059306078 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:00.418605 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ede7cd4d-6818-46c8-aa8c-01a059306078 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:00.419545 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ede7cd4d-6818-46c8-aa8c-01a059306078 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:00.420372 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ede7cd4d-6818-46c8-aa8c-01a059306078 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:00.420979 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ede7cd4d-6818-46c8-aa8c-01a059306078 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:00.421420 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ede7cd4d-6818-46c8-aa8c-01a059306078 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:00.443281 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.wsgi [None req-ede7cd4d-6818-46c8-aa8c-01a059306078 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] HTTP exception thrown: Instance 48b3d4bf-6b20-4327-8f8d-521417405254 could not be found. Aug 08 02:32:00.445601 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-ede7cd4d-6818-46c8-aa8c-01a059306078 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Request method failure captured: Aug 08 02:32:00.445601 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: request: GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254 HTTP/1.1 Aug 08 02:32:00.445601 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Accept: application/json Aug 08 02:32:00.445601 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Accept-Encoding: identity Aug 08 02:32:00.445601 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Connection: close Aug 08 02:32:00.445601 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Content-Length: 0 Aug 08 02:32:00.445601 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Content-Type: application/json Aug 08 02:32:00.445601 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Host: 10.222.0.27 Aug 08 02:32:00.445601 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Openstack-System-Scope: None Aug 08 02:32:00.445601 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: User-Agent: python-urllib3/1.26.11 Aug 08 02:32:00.445601 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Auth-Token: gAAAAABi8HQalNSG8U0tawdT8BYmZ3dO_Z_XJ7b_zGy3jhslHecNN8lPHCadrD9L1wE_fNDeX2KYU182-SQacEPppcF5Uwunh8fURGrax-nwHUph2qRhEJoRudpg1Feo8sNMetQ07fI55mt7EEM91lToca7t2N_CdvJ1rSwFvtzAdQ4uUEvan-8 Aug 08 02:32:00.445601 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Domain-Id: None Aug 08 02:32:00.445601 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Domain-Name: None Aug 08 02:32:00.445601 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Identity-Status: Confirmed Aug 08 02:32:00.445601 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Is-Admin-Project: True Aug 08 02:32:00.445601 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Domain-Id: default Aug 08 02:32:00.445601 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Domain-Name: Default Aug 08 02:32:00.445601 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Id: ee8baa7d6e294e27ae2fb12eded3709d Aug 08 02:32:00.445601 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Name: tempest-AttachVolumeTestJSON-827912071 Aug 08 02:32:00.445601 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Role: reader,member Aug 08 02:32:00.445601 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Roles: reader,member Aug 08 02:32:00.447517 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/ee8baa7d6e294e27ae2fb12eded3709d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/ee8baa7d6e294e27ae2fb12eded3709d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_ee8baa7d6e294e27ae2fb12eded3709d", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/ee8baa7d6e294e27ae2fb12eded3709d"}]}] Aug 08 02:32:00.447517 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant: tempest-AttachVolumeTestJSON-827912071 Aug 08 02:32:00.447517 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant-Id: ee8baa7d6e294e27ae2fb12eded3709d Aug 08 02:32:00.447517 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant-Name: tempest-AttachVolumeTestJSON-827912071 Aug 08 02:32:00.447517 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User: tempest-AttachVolumeTestJSON-827912071-project Aug 08 02:32:00.447517 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Domain-Id: default Aug 08 02:32:00.447517 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Domain-Name: Default Aug 08 02:32:00.447517 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Id: eb4c92edcb024e32a549650eb85f6952 Aug 08 02:32:00.447517 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Name: tempest-AttachVolumeTestJSON-827912071-project Aug 08 02:32:00.447517 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: method: > Aug 08 02:32:00.447517 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: exception: Instance 48b3d4bf-6b20-4327-8f8d-521417405254 could not be found. Aug 08 02:32:00.447517 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: action_args: {'id': '48b3d4bf-6b20-4327-8f8d-521417405254'} Aug 08 02:32:00.448308 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:32:00.448308 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:00.448308 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 08 02:32:00.448308 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 08 02:32:00.448308 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 08 02:32:00.448308 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 08 02:32:00.448308 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 08 02:32:00.448308 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 08 02:32:00.448308 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 08 02:32:00.448308 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 08 02:32:00.448308 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 48b3d4bf-6b20-4327-8f8d-521417405254 could not be found. Aug 08 02:32:00.448308 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:32:00.448308 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:32:00.448308 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:32:00.448308 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:00.448308 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:32:00.448308 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:32:00.448308 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:32:00.448308 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:32:00.448308 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:32:00.449709 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:32:00.449709 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 08 02:32:00.449709 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 08 02:32:00.449709 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 08 02:32:00.449709 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 08 02:32:00.449709 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 08 02:32:00.449709 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:32:00.449709 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 48b3d4bf-6b20-4327-8f8d-521417405254 could not be found. Aug 08 02:32:00.449709 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:32:00.449709 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:32:00.449709 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:32:00.449709 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:00.449709 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:32:00.449709 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:32:00.449709 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:32:00.449709 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:32:00.449709 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 48b3d4bf-6b20-4327-8f8d-521417405254 could not be found. Aug 08 02:32:00.449709 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:32:00.450829 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-ede7cd4d-6818-46c8-aa8c-01a059306078 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Returning 404 to user: Instance 48b3d4bf-6b20-4327-8f8d-521417405254 could not be found. {{(pid=118351) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:32:00.450829 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-ede7cd4d-6818-46c8-aa8c-01a059306078 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254" status: 404 len: 111 microversion: 2.1 time: 0.041677 Aug 08 02:32:00.450829 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 412/828] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:00 2022] GET /compute/v2.1/servers/48b3d4bf-6b20-4327-8f8d-521417405254 => generated 111 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 08 02:32:01.408132 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-80ea21cd-1b04-43b2-b582-fe3f55b35bfb tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:01.409566 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-80ea21cd-1b04-43b2-b582-fe3f55b35bfb tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:01.422175 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-80ea21cd-1b04-43b2-b582-fe3f55b35bfb tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:01.422915 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-80ea21cd-1b04-43b2-b582-fe3f55b35bfb tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:01.423555 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-80ea21cd-1b04-43b2-b582-fe3f55b35bfb tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:01.424303 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-80ea21cd-1b04-43b2-b582-fe3f55b35bfb tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:01.424789 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-80ea21cd-1b04-43b2-b582-fe3f55b35bfb tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:01.425578 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-80ea21cd-1b04-43b2-b582-fe3f55b35bfb tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:01.567949 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-80ea21cd-1b04-43b2-b582-fe3f55b35bfb tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:01.580429 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-80ea21cd-1b04-43b2-b582-fe3f55b35bfb tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:01.582310 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-80ea21cd-1b04-43b2-b582-fe3f55b35bfb tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:01.583806 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-80ea21cd-1b04-43b2-b582-fe3f55b35bfb tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1730 microversion: 2.20 time: 0.181754 Aug 08 02:32:01.584846 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 417/829] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:32:01 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1730 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:32:01.951490 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-883b57a5-e10b-4388-8fad-989979e63874 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:01.952828 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-883b57a5-e10b-4388-8fad-989979e63874 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:01.975115 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-883b57a5-e10b-4388-8fad-989979e63874 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:01.975899 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-883b57a5-e10b-4388-8fad-989979e63874 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:01.976687 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-883b57a5-e10b-4388-8fad-989979e63874 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:01.977625 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-883b57a5-e10b-4388-8fad-989979e63874 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:01.978251 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-883b57a5-e10b-4388-8fad-989979e63874 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:01.978843 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-883b57a5-e10b-4388-8fad-989979e63874 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:02.000889 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.wsgi [None req-883b57a5-e10b-4388-8fad-989979e63874 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] HTTP exception thrown: Instance ddd15bdd-8a1c-455b-91c9-504f69b5308a could not be found. Aug 08 02:32:02.002691 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-883b57a5-e10b-4388-8fad-989979e63874 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Request method failure captured: Aug 08 02:32:02.002691 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: request: DELETE /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a HTTP/1.1 Aug 08 02:32:02.002691 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Accept: application/json Aug 08 02:32:02.002691 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Accept-Encoding: identity Aug 08 02:32:02.002691 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Connection: close Aug 08 02:32:02.002691 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Content-Length: 0 Aug 08 02:32:02.002691 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Content-Type: application/json Aug 08 02:32:02.002691 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Host: 10.222.0.27 Aug 08 02:32:02.002691 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Openstack-System-Scope: None Aug 08 02:32:02.002691 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: User-Agent: python-urllib3/1.26.11 Aug 08 02:32:02.002691 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Auth-Token: gAAAAABi8HQalNSG8U0tawdT8BYmZ3dO_Z_XJ7b_zGy3jhslHecNN8lPHCadrD9L1wE_fNDeX2KYU182-SQacEPppcF5Uwunh8fURGrax-nwHUph2qRhEJoRudpg1Feo8sNMetQ07fI55mt7EEM91lToca7t2N_CdvJ1rSwFvtzAdQ4uUEvan-8 Aug 08 02:32:02.002691 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Domain-Id: None Aug 08 02:32:02.002691 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Domain-Name: None Aug 08 02:32:02.002691 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Identity-Status: Confirmed Aug 08 02:32:02.002691 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Is-Admin-Project: True Aug 08 02:32:02.002691 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Domain-Id: default Aug 08 02:32:02.002691 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Domain-Name: Default Aug 08 02:32:02.002691 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Id: ee8baa7d6e294e27ae2fb12eded3709d Aug 08 02:32:02.002691 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Name: tempest-AttachVolumeTestJSON-827912071 Aug 08 02:32:02.002691 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Role: reader,member Aug 08 02:32:02.002691 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Roles: reader,member Aug 08 02:32:02.004314 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/ee8baa7d6e294e27ae2fb12eded3709d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/ee8baa7d6e294e27ae2fb12eded3709d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_ee8baa7d6e294e27ae2fb12eded3709d", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/ee8baa7d6e294e27ae2fb12eded3709d"}]}] Aug 08 02:32:02.004314 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant: tempest-AttachVolumeTestJSON-827912071 Aug 08 02:32:02.004314 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant-Id: ee8baa7d6e294e27ae2fb12eded3709d Aug 08 02:32:02.004314 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant-Name: tempest-AttachVolumeTestJSON-827912071 Aug 08 02:32:02.004314 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User: tempest-AttachVolumeTestJSON-827912071-project Aug 08 02:32:02.004314 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Domain-Id: default Aug 08 02:32:02.004314 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Domain-Name: Default Aug 08 02:32:02.004314 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Id: eb4c92edcb024e32a549650eb85f6952 Aug 08 02:32:02.004314 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Name: tempest-AttachVolumeTestJSON-827912071-project Aug 08 02:32:02.004314 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: method: > Aug 08 02:32:02.004314 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: exception: Instance ddd15bdd-8a1c-455b-91c9-504f69b5308a could not be found. Aug 08 02:32:02.004314 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: action_args: {'id': 'ddd15bdd-8a1c-455b-91c9-504f69b5308a'} Aug 08 02:32:02.008163 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:32:02.008163 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:02.008163 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 08 02:32:02.008163 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 08 02:32:02.008163 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 08 02:32:02.008163 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 08 02:32:02.008163 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 08 02:32:02.008163 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 08 02:32:02.008163 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 08 02:32:02.008163 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 08 02:32:02.008163 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ddd15bdd-8a1c-455b-91c9-504f69b5308a could not be found. Aug 08 02:32:02.008163 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:32:02.008163 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:32:02.008163 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:32:02.008163 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:02.008163 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:32:02.008163 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:32:02.008163 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:32:02.008163 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:32:02.008163 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:32:02.009582 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:32:02.009582 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 08 02:32:02.009582 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 08 02:32:02.009582 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 08 02:32:02.009582 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 08 02:32:02.009582 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 08 02:32:02.009582 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 08 02:32:02.009582 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 08 02:32:02.009582 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:32:02.009582 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ddd15bdd-8a1c-455b-91c9-504f69b5308a could not be found. Aug 08 02:32:02.009582 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:32:02.009582 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:32:02.009582 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:32:02.009582 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:02.009582 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:32:02.009582 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:32:02.009582 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:32:02.009582 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:32:02.009582 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ddd15bdd-8a1c-455b-91c9-504f69b5308a could not be found. Aug 08 02:32:02.009582 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:32:02.010964 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-883b57a5-e10b-4388-8fad-989979e63874 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Returning 404 to user: Instance ddd15bdd-8a1c-455b-91c9-504f69b5308a could not be found. {{(pid=118351) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:32:02.010964 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-883b57a5-e10b-4388-8fad-989979e63874 tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "DELETE /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 404 len: 111 microversion: 2.1 time: 0.065625 Aug 08 02:32:02.010964 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 413/830] 10.222.0.27 () {58 vars in 1239 bytes} [Mon Aug 8 02:32:01 2022] DELETE /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 111 bytes in 67 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 08 02:32:02.013708 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-858c54d0-39c8-4145-91e3-099d25ccde8c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:02.014446 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-858c54d0-39c8-4145-91e3-099d25ccde8c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] [instance: ddd15bdd-8a1c-455b-91c9-504f69b5308a] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:02.021302 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-858c54d0-39c8-4145-91e3-099d25ccde8c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:02.021809 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-858c54d0-39c8-4145-91e3-099d25ccde8c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:02.022309 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-858c54d0-39c8-4145-91e3-099d25ccde8c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:02.023194 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-858c54d0-39c8-4145-91e3-099d25ccde8c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:02.023829 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-858c54d0-39c8-4145-91e3-099d25ccde8c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:02.024419 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-858c54d0-39c8-4145-91e3-099d25ccde8c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:02.040352 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.wsgi [None req-858c54d0-39c8-4145-91e3-099d25ccde8c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] HTTP exception thrown: Instance ddd15bdd-8a1c-455b-91c9-504f69b5308a could not be found. Aug 08 02:32:02.042170 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-858c54d0-39c8-4145-91e3-099d25ccde8c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Request method failure captured: Aug 08 02:32:02.042170 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: request: GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a HTTP/1.1 Aug 08 02:32:02.042170 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept: application/json Aug 08 02:32:02.042170 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept-Encoding: identity Aug 08 02:32:02.042170 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Connection: close Aug 08 02:32:02.042170 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Length: 0 Aug 08 02:32:02.042170 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Type: application/json Aug 08 02:32:02.042170 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Host: 10.222.0.27 Aug 08 02:32:02.042170 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Openstack-System-Scope: None Aug 08 02:32:02.042170 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: User-Agent: python-urllib3/1.26.11 Aug 08 02:32:02.042170 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Auth-Token: gAAAAABi8HQalNSG8U0tawdT8BYmZ3dO_Z_XJ7b_zGy3jhslHecNN8lPHCadrD9L1wE_fNDeX2KYU182-SQacEPppcF5Uwunh8fURGrax-nwHUph2qRhEJoRudpg1Feo8sNMetQ07fI55mt7EEM91lToca7t2N_CdvJ1rSwFvtzAdQ4uUEvan-8 Aug 08 02:32:02.042170 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Id: None Aug 08 02:32:02.042170 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Name: None Aug 08 02:32:02.042170 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Identity-Status: Confirmed Aug 08 02:32:02.042170 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Is-Admin-Project: True Aug 08 02:32:02.042170 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Id: default Aug 08 02:32:02.042170 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Name: Default Aug 08 02:32:02.042170 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Id: ee8baa7d6e294e27ae2fb12eded3709d Aug 08 02:32:02.042170 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Name: tempest-AttachVolumeTestJSON-827912071 Aug 08 02:32:02.042170 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Role: reader,member Aug 08 02:32:02.042170 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Roles: reader,member Aug 08 02:32:02.043901 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/ee8baa7d6e294e27ae2fb12eded3709d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/ee8baa7d6e294e27ae2fb12eded3709d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_ee8baa7d6e294e27ae2fb12eded3709d", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/ee8baa7d6e294e27ae2fb12eded3709d"}]}] Aug 08 02:32:02.043901 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant: tempest-AttachVolumeTestJSON-827912071 Aug 08 02:32:02.043901 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Id: ee8baa7d6e294e27ae2fb12eded3709d Aug 08 02:32:02.043901 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Name: tempest-AttachVolumeTestJSON-827912071 Aug 08 02:32:02.043901 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User: tempest-AttachVolumeTestJSON-827912071-project Aug 08 02:32:02.043901 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Id: default Aug 08 02:32:02.043901 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Name: Default Aug 08 02:32:02.043901 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Id: eb4c92edcb024e32a549650eb85f6952 Aug 08 02:32:02.043901 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Name: tempest-AttachVolumeTestJSON-827912071-project Aug 08 02:32:02.043901 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: method: > Aug 08 02:32:02.043901 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: exception: Instance ddd15bdd-8a1c-455b-91c9-504f69b5308a could not be found. Aug 08 02:32:02.043901 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: action_args: {'id': 'ddd15bdd-8a1c-455b-91c9-504f69b5308a'} Aug 08 02:32:02.045177 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:32:02.045177 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:02.045177 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 08 02:32:02.045177 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 08 02:32:02.045177 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 08 02:32:02.045177 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 08 02:32:02.045177 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 08 02:32:02.045177 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 08 02:32:02.045177 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 08 02:32:02.045177 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 08 02:32:02.045177 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ddd15bdd-8a1c-455b-91c9-504f69b5308a could not be found. Aug 08 02:32:02.045177 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:02.045177 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:32:02.045177 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:02.045177 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:02.045177 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:32:02.045177 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:32:02.045177 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:32:02.045177 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:32:02.045177 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:32:02.046712 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:32:02.046712 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 08 02:32:02.046712 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 08 02:32:02.046712 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 08 02:32:02.046712 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 08 02:32:02.046712 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 08 02:32:02.046712 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:32:02.046712 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ddd15bdd-8a1c-455b-91c9-504f69b5308a could not be found. Aug 08 02:32:02.046712 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:02.046712 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:32:02.046712 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:02.046712 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:02.046712 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:32:02.046712 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:32:02.046712 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:32:02.046712 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:32:02.046712 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ddd15bdd-8a1c-455b-91c9-504f69b5308a could not be found. Aug 08 02:32:02.046712 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:02.048394 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-858c54d0-39c8-4145-91e3-099d25ccde8c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] Returning 404 to user: Instance ddd15bdd-8a1c-455b-91c9-504f69b5308a could not be found. {{(pid=118352) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:32:02.048394 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-858c54d0-39c8-4145-91e3-099d25ccde8c tempest-AttachVolumeTestJSON-827912071 tempest-AttachVolumeTestJSON-827912071-project] 10.222.0.27 "GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a" status: 404 len: 111 microversion: 2.1 time: 0.037488 Aug 08 02:32:02.048394 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 418/831] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:02 2022] GET /compute/v2.1/servers/ddd15bdd-8a1c-455b-91c9-504f69b5308a => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 08 02:32:02.613110 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-e7ae867c-8998-4def-8817-02e13c0bba98 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:02.614236 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-e7ae867c-8998-4def-8817-02e13c0bba98 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:02.623236 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e7ae867c-8998-4def-8817-02e13c0bba98 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:02.624154 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e7ae867c-8998-4def-8817-02e13c0bba98 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:02.624801 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e7ae867c-8998-4def-8817-02e13c0bba98 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:02.625789 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e7ae867c-8998-4def-8817-02e13c0bba98 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:02.626601 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e7ae867c-8998-4def-8817-02e13c0bba98 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:02.627214 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e7ae867c-8998-4def-8817-02e13c0bba98 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:02.910393 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-e7ae867c-8998-4def-8817-02e13c0bba98 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:02.920091 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-e7ae867c-8998-4def-8817-02e13c0bba98 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:02.921995 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-e7ae867c-8998-4def-8817-02e13c0bba98 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:02.923404 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-e7ae867c-8998-4def-8817-02e13c0bba98 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1730 microversion: 2.20 time: 0.314073 Aug 08 02:32:02.924030 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 414/832] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:32:02 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1730 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:32:03.947499 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-3215f859-7948-42cf-8d3f-838e7a8d8de5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:03.948107 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-3215f859-7948-42cf-8d3f-838e7a8d8de5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:03.956356 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3215f859-7948-42cf-8d3f-838e7a8d8de5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:03.957134 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3215f859-7948-42cf-8d3f-838e7a8d8de5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:03.957754 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3215f859-7948-42cf-8d3f-838e7a8d8de5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:03.958619 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3215f859-7948-42cf-8d3f-838e7a8d8de5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:03.959246 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3215f859-7948-42cf-8d3f-838e7a8d8de5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:03.959675 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3215f859-7948-42cf-8d3f-838e7a8d8de5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:04.394298 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-3215f859-7948-42cf-8d3f-838e7a8d8de5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:04.404256 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-3215f859-7948-42cf-8d3f-838e7a8d8de5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:04.405790 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-3215f859-7948-42cf-8d3f-838e7a8d8de5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:04.407397 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-3215f859-7948-42cf-8d3f-838e7a8d8de5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1730 microversion: 2.20 time: 0.463940 Aug 08 02:32:04.425544 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 419/833] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:32:03 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1730 bytes in 482 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:32:05.432965 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-feeef88e-43a4-4fae-a6e2-de3d9e8db4c7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:05.439075 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-feeef88e-43a4-4fae-a6e2-de3d9e8db4c7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:05.446468 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-feeef88e-43a4-4fae-a6e2-de3d9e8db4c7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:05.446968 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-feeef88e-43a4-4fae-a6e2-de3d9e8db4c7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:05.450207 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-feeef88e-43a4-4fae-a6e2-de3d9e8db4c7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:05.451017 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-feeef88e-43a4-4fae-a6e2-de3d9e8db4c7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:05.451507 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-feeef88e-43a4-4fae-a6e2-de3d9e8db4c7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:05.451944 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-feeef88e-43a4-4fae-a6e2-de3d9e8db4c7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:05.642181 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-feeef88e-43a4-4fae-a6e2-de3d9e8db4c7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:05.656941 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-feeef88e-43a4-4fae-a6e2-de3d9e8db4c7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:05.661859 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-feeef88e-43a4-4fae-a6e2-de3d9e8db4c7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:05.663042 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-feeef88e-43a4-4fae-a6e2-de3d9e8db4c7 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1730 microversion: 2.20 time: 0.235643 Aug 08 02:32:05.663593 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 415/834] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:32:05 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1730 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:32:06.687468 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-077f7d67-4b3f-4efc-9fee-fad71200ca9e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:06.688211 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-077f7d67-4b3f-4efc-9fee-fad71200ca9e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:06.694909 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-077f7d67-4b3f-4efc-9fee-fad71200ca9e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:06.695465 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-077f7d67-4b3f-4efc-9fee-fad71200ca9e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:06.696234 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-077f7d67-4b3f-4efc-9fee-fad71200ca9e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:06.697037 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-077f7d67-4b3f-4efc-9fee-fad71200ca9e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:06.697568 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-077f7d67-4b3f-4efc-9fee-fad71200ca9e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:06.698058 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-077f7d67-4b3f-4efc-9fee-fad71200ca9e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:07.274416 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-077f7d67-4b3f-4efc-9fee-fad71200ca9e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:07.307707 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-077f7d67-4b3f-4efc-9fee-fad71200ca9e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:07.309911 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-077f7d67-4b3f-4efc-9fee-fad71200ca9e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:07.311329 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-077f7d67-4b3f-4efc-9fee-fad71200ca9e tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1730 microversion: 2.20 time: 0.628842 Aug 08 02:32:07.312369 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 420/835] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:32:06 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1730 bytes in 630 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:32:08.335592 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-caa0eef4-d3c9-46de-8a52-3a9578b2c71a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:08.336279 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-caa0eef4-d3c9-46de-8a52-3a9578b2c71a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:08.343315 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-caa0eef4-d3c9-46de-8a52-3a9578b2c71a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:08.343718 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-caa0eef4-d3c9-46de-8a52-3a9578b2c71a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:08.344148 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-caa0eef4-d3c9-46de-8a52-3a9578b2c71a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:08.344865 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-caa0eef4-d3c9-46de-8a52-3a9578b2c71a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:08.345310 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-caa0eef4-d3c9-46de-8a52-3a9578b2c71a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:08.345727 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-caa0eef4-d3c9-46de-8a52-3a9578b2c71a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:08.534832 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-caa0eef4-d3c9-46de-8a52-3a9578b2c71a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:08.543358 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-caa0eef4-d3c9-46de-8a52-3a9578b2c71a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:08.544903 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-caa0eef4-d3c9-46de-8a52-3a9578b2c71a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:08.546061 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-caa0eef4-d3c9-46de-8a52-3a9578b2c71a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1730 microversion: 2.20 time: 0.214466 Aug 08 02:32:08.546620 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 416/836] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:32:08 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1730 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:32:09.519765 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-1eb97504-7bfa-4f41-b9a0-2db9bc47464c tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:32:09.520285 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-1eb97504-7bfa-4f41-b9a0-2db9bc47464c tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:09.526756 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1eb97504-7bfa-4f41-b9a0-2db9bc47464c tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:09.527306 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1eb97504-7bfa-4f41-b9a0-2db9bc47464c tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:09.527841 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1eb97504-7bfa-4f41-b9a0-2db9bc47464c tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:09.528536 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1eb97504-7bfa-4f41-b9a0-2db9bc47464c tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:09.529195 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1eb97504-7bfa-4f41-b9a0-2db9bc47464c tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:09.532021 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1eb97504-7bfa-4f41-b9a0-2db9bc47464c tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:09.552470 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.objects.instance [None req-1eb97504-7bfa-4f41-b9a0-2db9bc47464c tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lazy-loading 'flavor' on Instance uuid 077e7935-3361-4ff5-9367-ba162efac09c {{(pid=118352) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 08 02:32:09.568489 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-2ae105d3-6291-4ccd-b2d3-8ff6c34f1832 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:09.569122 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-2ae105d3-6291-4ccd-b2d3-8ff6c34f1832 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:09.572500 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:09.577274 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2ae105d3-6291-4ccd-b2d3-8ff6c34f1832 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:09.577853 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2ae105d3-6291-4ccd-b2d3-8ff6c34f1832 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:09.578263 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2ae105d3-6291-4ccd-b2d3-8ff6c34f1832 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:09.578954 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2ae105d3-6291-4ccd-b2d3-8ff6c34f1832 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:09.579403 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2ae105d3-6291-4ccd-b2d3-8ff6c34f1832 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:09.580017 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2ae105d3-6291-4ccd-b2d3-8ff6c34f1832 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:09.739837 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-2ae105d3-6291-4ccd-b2d3-8ff6c34f1832 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:09.748976 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-2ae105d3-6291-4ccd-b2d3-8ff6c34f1832 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:09.754362 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-2ae105d3-6291-4ccd-b2d3-8ff6c34f1832 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:09.754362 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-2ae105d3-6291-4ccd-b2d3-8ff6c34f1832 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1730 microversion: 2.20 time: 0.187026 Aug 08 02:32:09.754362 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 417/837] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:32:09 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1730 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 08 02:32:09.955491 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-1eb97504-7bfa-4f41-b9a0-2db9bc47464c tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "POST /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c/action" status: 202 len: 0 microversion: 2.1 time: 0.439403 Aug 08 02:32:09.956101 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 421/838] 10.222.0.27 () {60 vars in 1278 bytes} [Mon Aug 8 02:32:09 2022] POST /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c/action => generated 0 bytes in 441 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 08 02:32:09.964615 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-2db029b9-777e-48fa-a580-37e72ab7b461 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:09.965335 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-2db029b9-777e-48fa-a580-37e72ab7b461 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:09.971421 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2db029b9-777e-48fa-a580-37e72ab7b461 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:09.971984 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2db029b9-777e-48fa-a580-37e72ab7b461 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:09.972489 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2db029b9-777e-48fa-a580-37e72ab7b461 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:09.973253 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2db029b9-777e-48fa-a580-37e72ab7b461 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:09.973739 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2db029b9-777e-48fa-a580-37e72ab7b461 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:09.974163 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2db029b9-777e-48fa-a580-37e72ab7b461 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:10.106125 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-2db029b9-777e-48fa-a580-37e72ab7b461 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:10.113781 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-2db029b9-777e-48fa-a580-37e72ab7b461 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1567 microversion: 2.1 time: 0.153327 Aug 08 02:32:10.114436 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 418/839] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:09 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1567 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:32:10.776606 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-c11b778e-9afa-4077-b9a8-0e9e6691c205 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:10.778336 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-c11b778e-9afa-4077-b9a8-0e9e6691c205 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:10.784831 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c11b778e-9afa-4077-b9a8-0e9e6691c205 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:10.785321 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c11b778e-9afa-4077-b9a8-0e9e6691c205 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:10.785765 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c11b778e-9afa-4077-b9a8-0e9e6691c205 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:10.786453 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c11b778e-9afa-4077-b9a8-0e9e6691c205 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:10.786908 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c11b778e-9afa-4077-b9a8-0e9e6691c205 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:10.787335 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c11b778e-9afa-4077-b9a8-0e9e6691c205 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:10.924079 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-c11b778e-9afa-4077-b9a8-0e9e6691c205 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:10.936164 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-c11b778e-9afa-4077-b9a8-0e9e6691c205 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:10.939387 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-c11b778e-9afa-4077-b9a8-0e9e6691c205 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:10.941270 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-c11b778e-9afa-4077-b9a8-0e9e6691c205 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1730 microversion: 2.20 time: 0.169030 Aug 08 02:32:10.952705 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 422/840] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:32:10 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1730 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:32:11.136223 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-51447ecd-1d34-46ff-ba4c-36f20af83128 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:11.137822 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-51447ecd-1d34-46ff-ba4c-36f20af83128 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:11.144305 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-51447ecd-1d34-46ff-ba4c-36f20af83128 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:11.144823 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-51447ecd-1d34-46ff-ba4c-36f20af83128 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:11.145259 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-51447ecd-1d34-46ff-ba4c-36f20af83128 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:11.145943 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-51447ecd-1d34-46ff-ba4c-36f20af83128 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:11.146496 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-51447ecd-1d34-46ff-ba4c-36f20af83128 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:11.146919 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-51447ecd-1d34-46ff-ba4c-36f20af83128 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:11.296737 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-51447ecd-1d34-46ff-ba4c-36f20af83128 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:11.306277 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-51447ecd-1d34-46ff-ba4c-36f20af83128 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1567 microversion: 2.1 time: 0.173971 Aug 08 02:32:11.306852 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 419/841] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:11 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1567 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:32:11.919544 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-d10e92ac-64e7-41e4-ba93-ea2cf19befee tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:11.920051 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-d10e92ac-64e7-41e4-ba93-ea2cf19befee tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: cc2b42f1-3dd0-4090-a905-93283047aea3] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:11.927094 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d10e92ac-64e7-41e4-ba93-ea2cf19befee tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:11.927583 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d10e92ac-64e7-41e4-ba93-ea2cf19befee tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:11.928020 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d10e92ac-64e7-41e4-ba93-ea2cf19befee tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:11.928734 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d10e92ac-64e7-41e4-ba93-ea2cf19befee tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:11.929191 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d10e92ac-64e7-41e4-ba93-ea2cf19befee tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:11.929666 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d10e92ac-64e7-41e4-ba93-ea2cf19befee tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:11.973835 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-f211fb3a-87b3-4a4f-8467-c5fb5bc854c0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:11.974448 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-f211fb3a-87b3-4a4f-8467-c5fb5bc854c0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:11.980761 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f211fb3a-87b3-4a4f-8467-c5fb5bc854c0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:11.982150 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f211fb3a-87b3-4a4f-8467-c5fb5bc854c0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:11.982150 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f211fb3a-87b3-4a4f-8467-c5fb5bc854c0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:11.982468 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f211fb3a-87b3-4a4f-8467-c5fb5bc854c0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:11.982631 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f211fb3a-87b3-4a4f-8467-c5fb5bc854c0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:11.983053 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f211fb3a-87b3-4a4f-8467-c5fb5bc854c0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:12.012602 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.objects.instance [None req-d10e92ac-64e7-41e4-ba93-ea2cf19befee tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lazy-loading 'services' on Instance uuid cc2b42f1-3dd0-4090-a905-93283047aea3 {{(pid=118352) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 08 02:32:12.120546 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-f211fb3a-87b3-4a4f-8467-c5fb5bc854c0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:12.129432 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-f211fb3a-87b3-4a4f-8467-c5fb5bc854c0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:12.131014 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-f211fb3a-87b3-4a4f-8467-c5fb5bc854c0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:12.132252 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-f211fb3a-87b3-4a4f-8467-c5fb5bc854c0 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1730 microversion: 2.20 time: 0.162239 Aug 08 02:32:12.132849 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 420/842] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:32:11 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1730 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:32:12.322835 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-d10e92ac-64e7-41e4-ba93-ea2cf19befee tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "DELETE /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3/os-volume_attachments/872d9b80-e690-4de4-8cff-d1e23d8de6cf" status: 202 len: 0 microversion: 2.1 time: 0.407226 Aug 08 02:32:12.323421 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 423/843] 10.222.0.27 () {58 vars in 1416 bytes} [Mon Aug 8 02:32:11 2022] DELETE /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3/os-volume_attachments/872d9b80-e690-4de4-8cff-d1e23d8de6cf => generated 0 bytes in 409 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 08 02:32:12.333879 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-f3d38e98-c2ef-47fb-aca7-148592aac70c tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:12.334869 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-3d396640-7386-42b2-9030-f14881064bce tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:12.335410 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-f3d38e98-c2ef-47fb-aca7-148592aac70c tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:12.338461 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-3d396640-7386-42b2-9030-f14881064bce tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: cc2b42f1-3dd0-4090-a905-93283047aea3] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:12.343490 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f3d38e98-c2ef-47fb-aca7-148592aac70c tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:12.343984 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f3d38e98-c2ef-47fb-aca7-148592aac70c tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:12.344418 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f3d38e98-c2ef-47fb-aca7-148592aac70c tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:12.345924 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f3d38e98-c2ef-47fb-aca7-148592aac70c tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:12.346384 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3d396640-7386-42b2-9030-f14881064bce tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:12.346767 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f3d38e98-c2ef-47fb-aca7-148592aac70c tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:12.347003 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3d396640-7386-42b2-9030-f14881064bce tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:12.347818 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3d396640-7386-42b2-9030-f14881064bce tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:12.348063 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f3d38e98-c2ef-47fb-aca7-148592aac70c tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:12.348301 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3d396640-7386-42b2-9030-f14881064bce tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:12.348935 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3d396640-7386-42b2-9030-f14881064bce tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:12.350871 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3d396640-7386-42b2-9030-f14881064bce tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:12.389098 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-3d396640-7386-42b2-9030-f14881064bce tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059392 Aug 08 02:32:12.389674 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 424/844] 10.222.0.27 () {58 vars in 1302 bytes} [Mon Aug 8 02:32:12 2022] GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:32:12.502829 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-f3d38e98-c2ef-47fb-aca7-148592aac70c tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:12.510305 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-f3d38e98-c2ef-47fb-aca7-148592aac70c tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1567 microversion: 2.1 time: 0.181212 Aug 08 02:32:12.510858 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 421/845] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:12 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1567 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:32:12.750527 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 08 02:32:12.750768 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Mon Aug 8 02:32:12 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 08 02:32:13.156299 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-0d5ab40d-9668-40e6-b70a-a5139fdfca9a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:13.156933 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-0d5ab40d-9668-40e6-b70a-a5139fdfca9a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:13.163279 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0d5ab40d-9668-40e6-b70a-a5139fdfca9a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:13.163762 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0d5ab40d-9668-40e6-b70a-a5139fdfca9a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:13.164197 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0d5ab40d-9668-40e6-b70a-a5139fdfca9a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:13.164908 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0d5ab40d-9668-40e6-b70a-a5139fdfca9a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:13.165374 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0d5ab40d-9668-40e6-b70a-a5139fdfca9a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:13.165842 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0d5ab40d-9668-40e6-b70a-a5139fdfca9a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:13.375375 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-0d5ab40d-9668-40e6-b70a-a5139fdfca9a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:13.384791 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-0d5ab40d-9668-40e6-b70a-a5139fdfca9a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:13.388463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-0d5ab40d-9668-40e6-b70a-a5139fdfca9a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:13.390205 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-0d5ab40d-9668-40e6-b70a-a5139fdfca9a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1730 microversion: 2.20 time: 0.238251 Aug 08 02:32:13.390807 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 425/846] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:32:13 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1730 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:32:13.402411 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-2f8cd919-62d1-4038-a3d9-fdf291a38f2b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:13.405282 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-2f8cd919-62d1-4038-a3d9-fdf291a38f2b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: cc2b42f1-3dd0-4090-a905-93283047aea3] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:13.414917 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2f8cd919-62d1-4038-a3d9-fdf291a38f2b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:13.416155 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2f8cd919-62d1-4038-a3d9-fdf291a38f2b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:13.416898 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2f8cd919-62d1-4038-a3d9-fdf291a38f2b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:13.417990 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2f8cd919-62d1-4038-a3d9-fdf291a38f2b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:13.418628 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2f8cd919-62d1-4038-a3d9-fdf291a38f2b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:13.419271 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2f8cd919-62d1-4038-a3d9-fdf291a38f2b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:13.454067 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-2f8cd919-62d1-4038-a3d9-fdf291a38f2b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.056313 Aug 08 02:32:13.460628 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 422/847] 10.222.0.27 () {58 vars in 1302 bytes} [Mon Aug 8 02:32:13 2022] GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3/os-volume_attachments => generated 25 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Aug 08 02:32:13.529913 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-dd36dcbc-7db2-4178-9314-2a0b411fdb5f tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:13.530466 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-dd36dcbc-7db2-4178-9314-2a0b411fdb5f tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: cc2b42f1-3dd0-4090-a905-93283047aea3] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:13.531163 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-482e8b31-5fea-49d2-bca1-5be179d277a3 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:13.533093 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-482e8b31-5fea-49d2-bca1-5be179d277a3 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:13.537469 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dd36dcbc-7db2-4178-9314-2a0b411fdb5f tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:13.537958 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dd36dcbc-7db2-4178-9314-2a0b411fdb5f tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:13.538392 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dd36dcbc-7db2-4178-9314-2a0b411fdb5f tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:13.539146 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dd36dcbc-7db2-4178-9314-2a0b411fdb5f tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:13.539281 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-482e8b31-5fea-49d2-bca1-5be179d277a3 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:13.539610 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dd36dcbc-7db2-4178-9314-2a0b411fdb5f tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:13.539728 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-482e8b31-5fea-49d2-bca1-5be179d277a3 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:13.540185 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dd36dcbc-7db2-4178-9314-2a0b411fdb5f tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:13.540330 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-482e8b31-5fea-49d2-bca1-5be179d277a3 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:13.540916 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-482e8b31-5fea-49d2-bca1-5be179d277a3 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:13.541370 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-482e8b31-5fea-49d2-bca1-5be179d277a3 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:13.541796 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-482e8b31-5fea-49d2-bca1-5be179d277a3 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:13.567268 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-dd36dcbc-7db2-4178-9314-2a0b411fdb5f tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: cc2b42f1-3dd0-4090-a905-93283047aea3] Going to try to terminate instance {{(pid=118352) delete /opt/stack/nova/nova/compute/api.py:2666}} Aug 08 02:32:13.713138 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-482e8b31-5fea-49d2-bca1-5be179d277a3 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:13.721733 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-482e8b31-5fea-49d2-bca1-5be179d277a3 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1567 microversion: 2.1 time: 0.194334 Aug 08 02:32:13.722671 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 423/848] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:13 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1567 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:32:14.001108 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-dd36dcbc-7db2-4178-9314-2a0b411fdb5f tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "DELETE /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3" status: 204 len: 0 microversion: 2.1 time: 0.474558 Aug 08 02:32:14.002144 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 426/849] 10.222.0.27 () {58 vars in 1239 bytes} [Mon Aug 8 02:32:13 2022] DELETE /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 08 02:32:14.011190 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-632d600a-56e1-45d5-a744-e7f1ae1c2603 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:14.012727 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-632d600a-56e1-45d5-a744-e7f1ae1c2603 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: cc2b42f1-3dd0-4090-a905-93283047aea3] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:14.020605 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-632d600a-56e1-45d5-a744-e7f1ae1c2603 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:14.021127 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-632d600a-56e1-45d5-a744-e7f1ae1c2603 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:14.021778 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-632d600a-56e1-45d5-a744-e7f1ae1c2603 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:14.022960 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-632d600a-56e1-45d5-a744-e7f1ae1c2603 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:14.023863 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-632d600a-56e1-45d5-a744-e7f1ae1c2603 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:14.024500 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-632d600a-56e1-45d5-a744-e7f1ae1c2603 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:14.212867 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-632d600a-56e1-45d5-a744-e7f1ae1c2603 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:14.223562 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-632d600a-56e1-45d5-a744-e7f1ae1c2603 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3" status: 200 len: 1682 microversion: 2.1 time: 0.216056 Aug 08 02:32:14.224186 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 424/850] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:14 2022] GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3 => generated 1682 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:32:14.416168 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-ab522891-28bd-4203-8c8b-becb502e95c4 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:14.417100 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-ab522891-28bd-4203-8c8b-becb502e95c4 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:14.425805 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ab522891-28bd-4203-8c8b-becb502e95c4 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:14.426297 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ab522891-28bd-4203-8c8b-becb502e95c4 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:14.426877 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ab522891-28bd-4203-8c8b-becb502e95c4 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:14.430129 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ab522891-28bd-4203-8c8b-becb502e95c4 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:14.430752 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ab522891-28bd-4203-8c8b-becb502e95c4 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:14.431258 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ab522891-28bd-4203-8c8b-becb502e95c4 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:14.586961 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-ab522891-28bd-4203-8c8b-becb502e95c4 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:14.596109 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-ab522891-28bd-4203-8c8b-becb502e95c4 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:14.597654 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-ab522891-28bd-4203-8c8b-becb502e95c4 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:14.599442 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-ab522891-28bd-4203-8c8b-becb502e95c4 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1730 microversion: 2.20 time: 0.187730 Aug 08 02:32:14.608880 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 427/851] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:32:14 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1730 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:32:14.745304 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-ac542508-6cf6-48fb-98c9-9de06e2d401a tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:14.746221 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-ac542508-6cf6-48fb-98c9-9de06e2d401a tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:14.746758 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [req-4e5e7bc2-295e-41b3-adeb-a53dbb4958e1 req-3cf0bb3b-0a1a-4f59-8fad-e88a9522dbe5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "946fd571-bd98-44f9-89f4-52f0116b642d", "name": "network-vif-plugged", "status": "completed", "tag": "0bf885ea-ca94-45b2-aafa-7461404e6cf2"}]} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:32:14.752974 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ac542508-6cf6-48fb-98c9-9de06e2d401a tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:14.753556 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ac542508-6cf6-48fb-98c9-9de06e2d401a tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:14.754198 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ac542508-6cf6-48fb-98c9-9de06e2d401a tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:14.754904 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ac542508-6cf6-48fb-98c9-9de06e2d401a tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:14.755357 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ac542508-6cf6-48fb-98c9-9de06e2d401a tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:14.755782 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ac542508-6cf6-48fb-98c9-9de06e2d401a tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:14.757484 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-4e5e7bc2-295e-41b3-adeb-a53dbb4958e1 req-3cf0bb3b-0a1a-4f59-8fad-e88a9522dbe5 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:14.758066 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-4e5e7bc2-295e-41b3-adeb-a53dbb4958e1 req-3cf0bb3b-0a1a-4f59-8fad-e88a9522dbe5 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:14.758495 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-4e5e7bc2-295e-41b3-adeb-a53dbb4958e1 req-3cf0bb3b-0a1a-4f59-8fad-e88a9522dbe5 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:14.770379 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.compute.server_external_events [req-4e5e7bc2-295e-41b3-adeb-a53dbb4958e1 req-3cf0bb3b-0a1a-4f59-8fad-e88a9522dbe5 service nova] Creating event network-vif-plugged:0bf885ea-ca94-45b2-aafa-7461404e6cf2 for instance 946fd571-bd98-44f9-89f4-52f0116b642d on hv-cinder-74014 Aug 08 02:32:14.775419 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [req-4e5e7bc2-295e-41b3-adeb-a53dbb4958e1 req-3cf0bb3b-0a1a-4f59-8fad-e88a9522dbe5 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032511 Aug 08 02:32:14.776199 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 428/852] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:32:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 08 02:32:14.895361 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-ac542508-6cf6-48fb-98c9-9de06e2d401a tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:14.904446 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-ac542508-6cf6-48fb-98c9-9de06e2d401a tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1567 microversion: 2.1 time: 0.163398 Aug 08 02:32:14.905792 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 425/853] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:14 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1567 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:32:15.252138 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-6de52c44-5a72-433c-b96e-01cf9e876e74 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:15.253123 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-6de52c44-5a72-433c-b96e-01cf9e876e74 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: cc2b42f1-3dd0-4090-a905-93283047aea3] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:15.259538 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6de52c44-5a72-433c-b96e-01cf9e876e74 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:15.264734 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6de52c44-5a72-433c-b96e-01cf9e876e74 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:15.264734 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6de52c44-5a72-433c-b96e-01cf9e876e74 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:15.264734 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6de52c44-5a72-433c-b96e-01cf9e876e74 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:15.264734 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6de52c44-5a72-433c-b96e-01cf9e876e74 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:15.265123 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6de52c44-5a72-433c-b96e-01cf9e876e74 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:15.410968 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-6de52c44-5a72-433c-b96e-01cf9e876e74 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:15.421198 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-6de52c44-5a72-433c-b96e-01cf9e876e74 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3" status: 200 len: 1682 microversion: 2.1 time: 0.173280 Aug 08 02:32:15.421840 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 429/854] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:15 2022] GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3 => generated 1682 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:32:15.623897 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-e98f407a-b778-43f5-80ee-889519dc8e0a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:15.624734 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-e98f407a-b778-43f5-80ee-889519dc8e0a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:15.633034 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e98f407a-b778-43f5-80ee-889519dc8e0a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:15.633815 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e98f407a-b778-43f5-80ee-889519dc8e0a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:15.634411 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e98f407a-b778-43f5-80ee-889519dc8e0a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:15.635160 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e98f407a-b778-43f5-80ee-889519dc8e0a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:15.635790 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e98f407a-b778-43f5-80ee-889519dc8e0a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:15.636307 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e98f407a-b778-43f5-80ee-889519dc8e0a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:15.774825 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-e98f407a-b778-43f5-80ee-889519dc8e0a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:15.783077 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-e98f407a-b778-43f5-80ee-889519dc8e0a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:15.784631 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-e98f407a-b778-43f5-80ee-889519dc8e0a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:15.785790 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-e98f407a-b778-43f5-80ee-889519dc8e0a tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1730 microversion: 2.20 time: 0.166862 Aug 08 02:32:15.786343 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 426/855] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:32:15 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1730 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:32:15.925860 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-c5d6c161-d4dd-492a-9c60-167bedfa6201 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:15.926468 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-c5d6c161-d4dd-492a-9c60-167bedfa6201 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:15.929059 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_api:SELECT=76,nova_api:DELETE=1,nova_api:UPDATE=2 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:15.934059 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c5d6c161-d4dd-492a-9c60-167bedfa6201 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:15.934540 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c5d6c161-d4dd-492a-9c60-167bedfa6201 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:15.934977 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c5d6c161-d4dd-492a-9c60-167bedfa6201 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:15.935665 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c5d6c161-d4dd-492a-9c60-167bedfa6201 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:15.936117 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c5d6c161-d4dd-492a-9c60-167bedfa6201 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:15.936537 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c5d6c161-d4dd-492a-9c60-167bedfa6201 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:16.083825 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-c5d6c161-d4dd-492a-9c60-167bedfa6201 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:16.092936 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-c5d6c161-d4dd-492a-9c60-167bedfa6201 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1567 microversion: 2.1 time: 0.170934 Aug 08 02:32:16.093501 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 430/856] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:15 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1567 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:32:16.444101 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-69c6970d-b3c3-4ed3-9d7e-bf4b5c30f12a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:16.444746 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-69c6970d-b3c3-4ed3-9d7e-bf4b5c30f12a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: cc2b42f1-3dd0-4090-a905-93283047aea3] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:16.451800 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-69c6970d-b3c3-4ed3-9d7e-bf4b5c30f12a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:16.452547 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-69c6970d-b3c3-4ed3-9d7e-bf4b5c30f12a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:16.453059 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-69c6970d-b3c3-4ed3-9d7e-bf4b5c30f12a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:16.453797 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-69c6970d-b3c3-4ed3-9d7e-bf4b5c30f12a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:16.454253 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-69c6970d-b3c3-4ed3-9d7e-bf4b5c30f12a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:16.454778 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-69c6970d-b3c3-4ed3-9d7e-bf4b5c30f12a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:16.523927 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-69c6970d-b3c3-4ed3-9d7e-bf4b5c30f12a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:16.535233 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-69c6970d-b3c3-4ed3-9d7e-bf4b5c30f12a tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3" status: 200 len: 1615 microversion: 2.1 time: 0.094299 Aug 08 02:32:16.535433 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [req-ded13ef6-67e0-445a-a99d-18a5af7cbd11 req-d6670ed1-84e9-4abd-b25e-66b66b45d366 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cc2b42f1-3dd0-4090-a905-93283047aea3", "name": "network-vif-deleted", "tag": "77fffbdc-6617-43da-9c07-c8e3bd46cfe3"}]} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:32:16.539379 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 427/857] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:16 2022] GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3 => generated 1615 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:32:16.546947 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-ded13ef6-67e0-445a-a99d-18a5af7cbd11 req-d6670ed1-84e9-4abd-b25e-66b66b45d366 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:16.547437 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-ded13ef6-67e0-445a-a99d-18a5af7cbd11 req-d6670ed1-84e9-4abd-b25e-66b66b45d366 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:16.547876 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-ded13ef6-67e0-445a-a99d-18a5af7cbd11 req-d6670ed1-84e9-4abd-b25e-66b66b45d366 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:16.562650 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.compute.server_external_events [req-ded13ef6-67e0-445a-a99d-18a5af7cbd11 req-d6670ed1-84e9-4abd-b25e-66b66b45d366 service nova] Creating event network-vif-deleted:77fffbdc-6617-43da-9c07-c8e3bd46cfe3 for instance cc2b42f1-3dd0-4090-a905-93283047aea3 on hv-cinder-74014 Aug 08 02:32:16.572795 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [req-ded13ef6-67e0-445a-a99d-18a5af7cbd11 req-d6670ed1-84e9-4abd-b25e-66b66b45d366 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041104 Aug 08 02:32:16.577973 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 431/858] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:32:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 08 02:32:16.811488 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-28f0c7c6-ac08-49cd-8b21-4ebc5ee156f2 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:16.813115 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-28f0c7c6-ac08-49cd-8b21-4ebc5ee156f2 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:16.819939 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-28f0c7c6-ac08-49cd-8b21-4ebc5ee156f2 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:16.820453 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-28f0c7c6-ac08-49cd-8b21-4ebc5ee156f2 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:16.820982 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-28f0c7c6-ac08-49cd-8b21-4ebc5ee156f2 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:16.821920 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-28f0c7c6-ac08-49cd-8b21-4ebc5ee156f2 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:16.822367 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-28f0c7c6-ac08-49cd-8b21-4ebc5ee156f2 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:16.822960 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-28f0c7c6-ac08-49cd-8b21-4ebc5ee156f2 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:16.963042 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-28f0c7c6-ac08-49cd-8b21-4ebc5ee156f2 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:16.972098 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-28f0c7c6-ac08-49cd-8b21-4ebc5ee156f2 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:16.973667 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-28f0c7c6-ac08-49cd-8b21-4ebc5ee156f2 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:16.974845 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-28f0c7c6-ac08-49cd-8b21-4ebc5ee156f2 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1730 microversion: 2.20 time: 0.168285 Aug 08 02:32:16.975448 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 428/859] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:32:16 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1730 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:32:17.116437 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-c6fb05b9-f2de-4bc6-bbf4-97e9112a6e62 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:17.117265 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-c6fb05b9-f2de-4bc6-bbf4-97e9112a6e62 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:17.125022 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c6fb05b9-f2de-4bc6-bbf4-97e9112a6e62 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:17.130962 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c6fb05b9-f2de-4bc6-bbf4-97e9112a6e62 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:17.130962 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c6fb05b9-f2de-4bc6-bbf4-97e9112a6e62 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:17.130962 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c6fb05b9-f2de-4bc6-bbf4-97e9112a6e62 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:17.130962 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c6fb05b9-f2de-4bc6-bbf4-97e9112a6e62 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:17.131384 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c6fb05b9-f2de-4bc6-bbf4-97e9112a6e62 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:17.270698 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-c6fb05b9-f2de-4bc6-bbf4-97e9112a6e62 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:17.279008 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-c6fb05b9-f2de-4bc6-bbf4-97e9112a6e62 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1567 microversion: 2.1 time: 0.167690 Aug 08 02:32:17.279702 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 432/860] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:17 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1567 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:32:17.558967 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-76a87c1e-dd97-42e3-94bd-6e036e1b9d5c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:17.559858 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-76a87c1e-dd97-42e3-94bd-6e036e1b9d5c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: cc2b42f1-3dd0-4090-a905-93283047aea3] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:17.566042 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-76a87c1e-dd97-42e3-94bd-6e036e1b9d5c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:17.566525 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-76a87c1e-dd97-42e3-94bd-6e036e1b9d5c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:17.566999 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-76a87c1e-dd97-42e3-94bd-6e036e1b9d5c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:17.567632 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-76a87c1e-dd97-42e3-94bd-6e036e1b9d5c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:17.568068 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-76a87c1e-dd97-42e3-94bd-6e036e1b9d5c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:17.568492 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-76a87c1e-dd97-42e3-94bd-6e036e1b9d5c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:17.637127 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-76a87c1e-dd97-42e3-94bd-6e036e1b9d5c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:17.639543 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell1:SELECT=100 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:17.646545 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-76a87c1e-dd97-42e3-94bd-6e036e1b9d5c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3" status: 200 len: 1326 microversion: 2.1 time: 0.092225 Aug 08 02:32:17.647378 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 429/861] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:17 2022] GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3 => generated 1326 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:32:18.000091 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-36aeef53-8e6e-46c3-b83b-9ee321ef85d8 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:18.000718 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-36aeef53-8e6e-46c3-b83b-9ee321ef85d8 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:18.006689 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-36aeef53-8e6e-46c3-b83b-9ee321ef85d8 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:18.007163 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-36aeef53-8e6e-46c3-b83b-9ee321ef85d8 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:18.007868 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-36aeef53-8e6e-46c3-b83b-9ee321ef85d8 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:18.008556 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-36aeef53-8e6e-46c3-b83b-9ee321ef85d8 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:18.009033 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-36aeef53-8e6e-46c3-b83b-9ee321ef85d8 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:18.009452 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-36aeef53-8e6e-46c3-b83b-9ee321ef85d8 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:18.140567 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-36aeef53-8e6e-46c3-b83b-9ee321ef85d8 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:18.148998 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-36aeef53-8e6e-46c3-b83b-9ee321ef85d8 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:18.150572 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-36aeef53-8e6e-46c3-b83b-9ee321ef85d8 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:18.151724 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-36aeef53-8e6e-46c3-b83b-9ee321ef85d8 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1730 microversion: 2.20 time: 0.156778 Aug 08 02:32:18.153363 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 433/862] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:32:17 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1730 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:32:18.299720 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-77114118-4c5f-4356-9daa-7de6270d9d20 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:18.300338 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-77114118-4c5f-4356-9daa-7de6270d9d20 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:18.307284 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-77114118-4c5f-4356-9daa-7de6270d9d20 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:18.307780 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-77114118-4c5f-4356-9daa-7de6270d9d20 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:18.308358 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-77114118-4c5f-4356-9daa-7de6270d9d20 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:18.309100 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-77114118-4c5f-4356-9daa-7de6270d9d20 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:18.309558 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-77114118-4c5f-4356-9daa-7de6270d9d20 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:18.309987 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-77114118-4c5f-4356-9daa-7de6270d9d20 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:18.446102 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-77114118-4c5f-4356-9daa-7de6270d9d20 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:18.450791 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-77114118-4c5f-4356-9daa-7de6270d9d20 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1567 microversion: 2.1 time: 0.155014 Aug 08 02:32:18.451396 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 430/863] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:18 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1567 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:32:18.668964 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-350a9028-05ed-4d06-97bc-cd111adcff45 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:18.669570 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-350a9028-05ed-4d06-97bc-cd111adcff45 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: cc2b42f1-3dd0-4090-a905-93283047aea3] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:18.675903 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-350a9028-05ed-4d06-97bc-cd111adcff45 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:18.676383 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-350a9028-05ed-4d06-97bc-cd111adcff45 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:18.676851 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-350a9028-05ed-4d06-97bc-cd111adcff45 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:18.677541 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-350a9028-05ed-4d06-97bc-cd111adcff45 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:18.677990 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-350a9028-05ed-4d06-97bc-cd111adcff45 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:18.678466 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-350a9028-05ed-4d06-97bc-cd111adcff45 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:18.740519 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-350a9028-05ed-4d06-97bc-cd111adcff45 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:18.748974 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-350a9028-05ed-4d06-97bc-cd111adcff45 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3" status: 200 len: 1326 microversion: 2.1 time: 0.084614 Aug 08 02:32:18.749537 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 434/864] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:18 2022] GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3 => generated 1326 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:32:19.181004 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-78fbf732-3f28-41ba-ad74-28107e364237 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:19.182118 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-78fbf732-3f28-41ba-ad74-28107e364237 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:19.193931 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-78fbf732-3f28-41ba-ad74-28107e364237 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:19.268082 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-78fbf732-3f28-41ba-ad74-28107e364237 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:19.268082 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-78fbf732-3f28-41ba-ad74-28107e364237 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:19.268082 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-78fbf732-3f28-41ba-ad74-28107e364237 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:19.268082 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-78fbf732-3f28-41ba-ad74-28107e364237 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:19.269183 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-78fbf732-3f28-41ba-ad74-28107e364237 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:19.341863 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-78fbf732-3f28-41ba-ad74-28107e364237 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:19.349911 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-78fbf732-3f28-41ba-ad74-28107e364237 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:19.351410 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-78fbf732-3f28-41ba-ad74-28107e364237 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:19.352553 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-78fbf732-3f28-41ba-ad74-28107e364237 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1730 microversion: 2.20 time: 0.179309 Aug 08 02:32:19.353348 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 431/865] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:32:19 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1730 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:32:19.472145 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-81e208fc-ad6e-45ff-8aef-19910b50df0d tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:19.472785 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-81e208fc-ad6e-45ff-8aef-19910b50df0d tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:19.479319 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-81e208fc-ad6e-45ff-8aef-19910b50df0d tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:19.479798 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-81e208fc-ad6e-45ff-8aef-19910b50df0d tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:19.480579 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-81e208fc-ad6e-45ff-8aef-19910b50df0d tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:19.481373 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-81e208fc-ad6e-45ff-8aef-19910b50df0d tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:19.481830 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-81e208fc-ad6e-45ff-8aef-19910b50df0d tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:19.482252 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-81e208fc-ad6e-45ff-8aef-19910b50df0d tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:19.627146 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-81e208fc-ad6e-45ff-8aef-19910b50df0d tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:19.630720 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell1:UPDATE=3,nova_cell1:SELECT=94,nova_cell1:INSERT=3 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:19.638802 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-81e208fc-ad6e-45ff-8aef-19910b50df0d tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1567 microversion: 2.1 time: 0.170501 Aug 08 02:32:19.639389 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 435/866] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:19 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1567 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:32:19.769714 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-c08becd4-9061-4250-ba45-2e8154022e88 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:19.772375 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-c08becd4-9061-4250-ba45-2e8154022e88 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: cc2b42f1-3dd0-4090-a905-93283047aea3] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:19.776439 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c08becd4-9061-4250-ba45-2e8154022e88 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:19.777340 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c08becd4-9061-4250-ba45-2e8154022e88 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:19.777533 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c08becd4-9061-4250-ba45-2e8154022e88 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:19.778139 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c08becd4-9061-4250-ba45-2e8154022e88 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:19.778589 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c08becd4-9061-4250-ba45-2e8154022e88 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:19.779006 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c08becd4-9061-4250-ba45-2e8154022e88 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:19.859378 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-c08becd4-9061-4250-ba45-2e8154022e88 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578cd837d9fe442587f8c6c8528334c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:19.866814 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-c08becd4-9061-4250-ba45-2e8154022e88 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3" status: 200 len: 1326 microversion: 2.1 time: 0.100862 Aug 08 02:32:19.868401 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 432/867] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:19 2022] GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3 => generated 1326 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:32:20.378435 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-e53b4486-50bb-4822-b54e-c14a8c071d87 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:20.378962 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-e53b4486-50bb-4822-b54e-c14a8c071d87 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:20.385241 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e53b4486-50bb-4822-b54e-c14a8c071d87 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:20.385736 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e53b4486-50bb-4822-b54e-c14a8c071d87 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:20.386289 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e53b4486-50bb-4822-b54e-c14a8c071d87 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:20.387020 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e53b4486-50bb-4822-b54e-c14a8c071d87 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:20.387484 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e53b4486-50bb-4822-b54e-c14a8c071d87 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:20.387915 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e53b4486-50bb-4822-b54e-c14a8c071d87 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:20.556732 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-e53b4486-50bb-4822-b54e-c14a8c071d87 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:20.574834 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-e53b4486-50bb-4822-b54e-c14a8c071d87 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:20.579306 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-e53b4486-50bb-4822-b54e-c14a8c071d87 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:20.584093 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-e53b4486-50bb-4822-b54e-c14a8c071d87 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1730 microversion: 2.20 time: 0.209947 Aug 08 02:32:20.604978 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 436/868] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:32:20 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1730 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:32:20.660066 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-b5201eeb-019e-4a4d-89b1-a4a57a36b9f6 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:20.660692 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-b5201eeb-019e-4a4d-89b1-a4a57a36b9f6 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:20.667122 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b5201eeb-019e-4a4d-89b1-a4a57a36b9f6 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:20.668216 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b5201eeb-019e-4a4d-89b1-a4a57a36b9f6 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:20.668621 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b5201eeb-019e-4a4d-89b1-a4a57a36b9f6 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:20.669545 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b5201eeb-019e-4a4d-89b1-a4a57a36b9f6 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:20.670019 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b5201eeb-019e-4a4d-89b1-a4a57a36b9f6 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:20.678061 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b5201eeb-019e-4a4d-89b1-a4a57a36b9f6 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:20.809170 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-b5201eeb-019e-4a4d-89b1-a4a57a36b9f6 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:20.817662 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-b5201eeb-019e-4a4d-89b1-a4a57a36b9f6 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1567 microversion: 2.1 time: 0.161355 Aug 08 02:32:20.818345 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 433/869] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:20 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1567 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:32:20.888515 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-c1a558ae-701e-423d-a31c-cf8cace28783 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:20.889156 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-c1a558ae-701e-423d-a31c-cf8cace28783 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: cc2b42f1-3dd0-4090-a905-93283047aea3] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:20.896053 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c1a558ae-701e-423d-a31c-cf8cace28783 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:20.896541 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c1a558ae-701e-423d-a31c-cf8cace28783 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:20.897033 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c1a558ae-701e-423d-a31c-cf8cace28783 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:20.897734 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c1a558ae-701e-423d-a31c-cf8cace28783 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:20.898204 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c1a558ae-701e-423d-a31c-cf8cace28783 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:20.898635 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c1a558ae-701e-423d-a31c-cf8cace28783 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:20.913662 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.wsgi [None req-c1a558ae-701e-423d-a31c-cf8cace28783 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] HTTP exception thrown: Instance cc2b42f1-3dd0-4090-a905-93283047aea3 could not be found. Aug 08 02:32:20.915294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-c1a558ae-701e-423d-a31c-cf8cace28783 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Request method failure captured: Aug 08 02:32:20.915294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: request: GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3 HTTP/1.1 Aug 08 02:32:20.915294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept: application/json Aug 08 02:32:20.915294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept-Encoding: identity Aug 08 02:32:20.915294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Connection: close Aug 08 02:32:20.915294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Length: 0 Aug 08 02:32:20.915294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Type: application/json Aug 08 02:32:20.915294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Host: 10.222.0.27 Aug 08 02:32:20.915294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Openstack-System-Scope: None Aug 08 02:32:20.915294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: User-Agent: python-urllib3/1.26.11 Aug 08 02:32:20.915294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Auth-Token: gAAAAABi8HQUs7QI_wjv0vSDVlijMnZ9-Lkgqc--Bu80jwWL-CoTBjbpe1sae_INrRIGeFqgwK-ezXnEOgqpG2NO5P97SEgPcY4zY_QgnaTA6xv-rzef5kDF5s4pU5M8QdM-T8eUF020_v3zsmgR78rxXaglg0C5QuxYvT8R4THTJZuwa_f26Bo Aug 08 02:32:20.915294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Id: None Aug 08 02:32:20.915294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Name: None Aug 08 02:32:20.915294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Identity-Status: Confirmed Aug 08 02:32:20.915294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Is-Admin-Project: True Aug 08 02:32:20.915294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Id: default Aug 08 02:32:20.915294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Name: Default Aug 08 02:32:20.915294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Id: ca29b2b9a59d4826bfd25f4dcfa9cffe Aug 08 02:32:20.915294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Name: tempest-AttachVolumeNegativeTest-420450256 Aug 08 02:32:20.915294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Role: reader,member Aug 08 02:32:20.915294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Roles: reader,member Aug 08 02:32:20.916753 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/ca29b2b9a59d4826bfd25f4dcfa9cffe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/ca29b2b9a59d4826bfd25f4dcfa9cffe"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_ca29b2b9a59d4826bfd25f4dcfa9cffe", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/ca29b2b9a59d4826bfd25f4dcfa9cffe"}]}] Aug 08 02:32:20.916753 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant: tempest-AttachVolumeNegativeTest-420450256 Aug 08 02:32:20.916753 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Id: ca29b2b9a59d4826bfd25f4dcfa9cffe Aug 08 02:32:20.916753 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-420450256 Aug 08 02:32:20.916753 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User: tempest-AttachVolumeNegativeTest-420450256-project Aug 08 02:32:20.916753 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Id: default Aug 08 02:32:20.916753 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Name: Default Aug 08 02:32:20.916753 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Id: 578cd837d9fe442587f8c6c8528334c0 Aug 08 02:32:20.916753 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Name: tempest-AttachVolumeNegativeTest-420450256-project Aug 08 02:32:20.916753 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: method: > Aug 08 02:32:20.916753 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: exception: Instance cc2b42f1-3dd0-4090-a905-93283047aea3 could not be found. Aug 08 02:32:20.916753 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: action_args: {'id': 'cc2b42f1-3dd0-4090-a905-93283047aea3'} Aug 08 02:32:20.918900 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:32:20.918900 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:20.918900 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 08 02:32:20.918900 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 08 02:32:20.918900 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 08 02:32:20.918900 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 08 02:32:20.918900 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 08 02:32:20.918900 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 08 02:32:20.918900 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 08 02:32:20.918900 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 08 02:32:20.918900 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance cc2b42f1-3dd0-4090-a905-93283047aea3 could not be found. Aug 08 02:32:20.918900 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:20.918900 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:32:20.918900 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:20.918900 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:20.918900 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:32:20.918900 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:32:20.918900 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:32:20.918900 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:32:20.918900 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:32:20.921596 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:32:20.921596 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 08 02:32:20.921596 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 08 02:32:20.921596 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 08 02:32:20.921596 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 08 02:32:20.921596 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 08 02:32:20.921596 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:32:20.921596 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance cc2b42f1-3dd0-4090-a905-93283047aea3 could not be found. Aug 08 02:32:20.921596 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:20.921596 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:32:20.921596 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:20.921596 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:20.921596 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:32:20.921596 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:32:20.921596 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:32:20.921596 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:32:20.921596 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance cc2b42f1-3dd0-4090-a905-93283047aea3 could not be found. Aug 08 02:32:20.921596 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:20.923626 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-c1a558ae-701e-423d-a31c-cf8cace28783 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Returning 404 to user: Instance cc2b42f1-3dd0-4090-a905-93283047aea3 could not be found. {{(pid=118352) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:32:20.923626 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-c1a558ae-701e-423d-a31c-cf8cace28783 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3" status: 404 len: 111 microversion: 2.1 time: 0.032448 Aug 08 02:32:20.923626 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 437/870] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:20 2022] GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 08 02:32:20.926232 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-e25cd682-16e6-4b06-8b41-6d498a88d50b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '.version_select at 0x7efc6f7d1310>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:21.048253 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-e25cd682-16e6-4b06-8b41-6d498a88d50b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1786670406" status: 202 len: 0 microversion: 2.1 time: 0.125754 Aug 08 02:32:21.048911 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 434/871] 10.222.0.27 () {58 vars in 1221 bytes} [Mon Aug 8 02:32:20 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1786670406 => generated 0 bytes in 127 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 08 02:32:21.617276 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-ce4aa5ae-770b-433c-8c2b-1ed9df90ba1f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:21.617891 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-ce4aa5ae-770b-433c-8c2b-1ed9df90ba1f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:21.633361 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ce4aa5ae-770b-433c-8c2b-1ed9df90ba1f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:21.633361 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ce4aa5ae-770b-433c-8c2b-1ed9df90ba1f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:21.633361 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ce4aa5ae-770b-433c-8c2b-1ed9df90ba1f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:21.633361 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ce4aa5ae-770b-433c-8c2b-1ed9df90ba1f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:21.633684 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ce4aa5ae-770b-433c-8c2b-1ed9df90ba1f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:21.633684 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ce4aa5ae-770b-433c-8c2b-1ed9df90ba1f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:21.841550 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-7777bb2c-8dd2-4e13-a615-7e261cfdab31 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:21.842622 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-7777bb2c-8dd2-4e13-a615-7e261cfdab31 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:21.848577 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7777bb2c-8dd2-4e13-a615-7e261cfdab31 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:21.849241 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7777bb2c-8dd2-4e13-a615-7e261cfdab31 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:21.849680 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7777bb2c-8dd2-4e13-a615-7e261cfdab31 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:21.850369 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7777bb2c-8dd2-4e13-a615-7e261cfdab31 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:21.850900 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7777bb2c-8dd2-4e13-a615-7e261cfdab31 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:21.851481 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7777bb2c-8dd2-4e13-a615-7e261cfdab31 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:21.854662 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-ce4aa5ae-770b-433c-8c2b-1ed9df90ba1f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:21.877628 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-ce4aa5ae-770b-433c-8c2b-1ed9df90ba1f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:21.879378 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-ce4aa5ae-770b-433c-8c2b-1ed9df90ba1f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:21.884727 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-ce4aa5ae-770b-433c-8c2b-1ed9df90ba1f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1730 microversion: 2.20 time: 0.268705 Aug 08 02:32:21.884727 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 438/872] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:32:21 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1730 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:32:21.993005 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-7777bb2c-8dd2-4e13-a615-7e261cfdab31 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:22.001335 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-7777bb2c-8dd2-4e13-a615-7e261cfdab31 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1567 microversion: 2.1 time: 0.165869 Aug 08 02:32:22.001902 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 435/873] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:21 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1567 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:32:22.906959 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-346558ca-a45b-4980-9594-9f7b5f5dae0d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:22.907612 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-346558ca-a45b-4980-9594-9f7b5f5dae0d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:22.914075 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-346558ca-a45b-4980-9594-9f7b5f5dae0d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:22.914603 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-346558ca-a45b-4980-9594-9f7b5f5dae0d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:22.915032 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-346558ca-a45b-4980-9594-9f7b5f5dae0d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:22.915710 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-346558ca-a45b-4980-9594-9f7b5f5dae0d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:22.916163 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-346558ca-a45b-4980-9594-9f7b5f5dae0d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:22.916584 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-346558ca-a45b-4980-9594-9f7b5f5dae0d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:23.022934 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-f15bbafb-18a5-415d-8852-e8d6ae279f22 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:23.023599 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-f15bbafb-18a5-415d-8852-e8d6ae279f22 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:23.026076 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_api:SELECT=61,nova_api:DELETE=1 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:23.031306 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f15bbafb-18a5-415d-8852-e8d6ae279f22 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:23.031790 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f15bbafb-18a5-415d-8852-e8d6ae279f22 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:23.032220 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f15bbafb-18a5-415d-8852-e8d6ae279f22 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:23.032935 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f15bbafb-18a5-415d-8852-e8d6ae279f22 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:23.033387 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f15bbafb-18a5-415d-8852-e8d6ae279f22 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:23.033806 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f15bbafb-18a5-415d-8852-e8d6ae279f22 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:23.053148 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-346558ca-a45b-4980-9594-9f7b5f5dae0d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:23.061920 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-346558ca-a45b-4980-9594-9f7b5f5dae0d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:23.063428 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-346558ca-a45b-4980-9594-9f7b5f5dae0d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:23.064583 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-346558ca-a45b-4980-9594-9f7b5f5dae0d tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1717 microversion: 2.20 time: 0.161642 Aug 08 02:32:23.065194 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 439/874] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:32:22 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1717 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:32:23.177499 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-f15bbafb-18a5-415d-8852-e8d6ae279f22 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:23.185770 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-f15bbafb-18a5-415d-8852-e8d6ae279f22 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1567 microversion: 2.1 time: 0.166633 Aug 08 02:32:23.186394 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 436/875] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:23 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1567 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:32:23.193676 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-b4401e42-439c-4744-8ed4-0d7b558f1ab7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:23.194280 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-b4401e42-439c-4744-8ed4-0d7b558f1ab7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: cc2b42f1-3dd0-4090-a905-93283047aea3] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:23.200987 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b4401e42-439c-4744-8ed4-0d7b558f1ab7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:23.201470 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b4401e42-439c-4744-8ed4-0d7b558f1ab7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:23.204818 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b4401e42-439c-4744-8ed4-0d7b558f1ab7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:23.204818 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b4401e42-439c-4744-8ed4-0d7b558f1ab7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:23.204818 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b4401e42-439c-4744-8ed4-0d7b558f1ab7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:23.204818 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b4401e42-439c-4744-8ed4-0d7b558f1ab7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:23.217051 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.wsgi [None req-b4401e42-439c-4744-8ed4-0d7b558f1ab7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] HTTP exception thrown: Instance cc2b42f1-3dd0-4090-a905-93283047aea3 could not be found. Aug 08 02:32:23.218583 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-b4401e42-439c-4744-8ed4-0d7b558f1ab7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Request method failure captured: Aug 08 02:32:23.218583 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: request: DELETE /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3 HTTP/1.1 Aug 08 02:32:23.218583 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept: application/json Aug 08 02:32:23.218583 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept-Encoding: identity Aug 08 02:32:23.218583 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Connection: close Aug 08 02:32:23.218583 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Length: 0 Aug 08 02:32:23.218583 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Type: application/json Aug 08 02:32:23.218583 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Host: 10.222.0.27 Aug 08 02:32:23.218583 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Openstack-System-Scope: None Aug 08 02:32:23.218583 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: User-Agent: python-urllib3/1.26.11 Aug 08 02:32:23.218583 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Auth-Token: gAAAAABi8HQUs7QI_wjv0vSDVlijMnZ9-Lkgqc--Bu80jwWL-CoTBjbpe1sae_INrRIGeFqgwK-ezXnEOgqpG2NO5P97SEgPcY4zY_QgnaTA6xv-rzef5kDF5s4pU5M8QdM-T8eUF020_v3zsmgR78rxXaglg0C5QuxYvT8R4THTJZuwa_f26Bo Aug 08 02:32:23.218583 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Id: None Aug 08 02:32:23.218583 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Name: None Aug 08 02:32:23.218583 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Identity-Status: Confirmed Aug 08 02:32:23.218583 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Is-Admin-Project: True Aug 08 02:32:23.218583 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Id: default Aug 08 02:32:23.218583 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Name: Default Aug 08 02:32:23.218583 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Id: ca29b2b9a59d4826bfd25f4dcfa9cffe Aug 08 02:32:23.218583 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Name: tempest-AttachVolumeNegativeTest-420450256 Aug 08 02:32:23.218583 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Role: reader,member Aug 08 02:32:23.218583 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Roles: reader,member Aug 08 02:32:23.220154 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/ca29b2b9a59d4826bfd25f4dcfa9cffe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/ca29b2b9a59d4826bfd25f4dcfa9cffe"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_ca29b2b9a59d4826bfd25f4dcfa9cffe", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/ca29b2b9a59d4826bfd25f4dcfa9cffe"}]}] Aug 08 02:32:23.220154 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant: tempest-AttachVolumeNegativeTest-420450256 Aug 08 02:32:23.220154 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Id: ca29b2b9a59d4826bfd25f4dcfa9cffe Aug 08 02:32:23.220154 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-420450256 Aug 08 02:32:23.220154 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User: tempest-AttachVolumeNegativeTest-420450256-project Aug 08 02:32:23.220154 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Id: default Aug 08 02:32:23.220154 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Name: Default Aug 08 02:32:23.220154 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Id: 578cd837d9fe442587f8c6c8528334c0 Aug 08 02:32:23.220154 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Name: tempest-AttachVolumeNegativeTest-420450256-project Aug 08 02:32:23.220154 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: method: > Aug 08 02:32:23.220154 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: exception: Instance cc2b42f1-3dd0-4090-a905-93283047aea3 could not be found. Aug 08 02:32:23.220154 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: action_args: {'id': 'cc2b42f1-3dd0-4090-a905-93283047aea3'} Aug 08 02:32:23.221110 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:32:23.221110 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:23.221110 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 08 02:32:23.221110 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 08 02:32:23.221110 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 08 02:32:23.221110 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 08 02:32:23.221110 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 08 02:32:23.221110 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 08 02:32:23.221110 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 08 02:32:23.221110 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 08 02:32:23.221110 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance cc2b42f1-3dd0-4090-a905-93283047aea3 could not be found. Aug 08 02:32:23.221110 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:23.221110 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:32:23.221110 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:23.221110 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:23.221110 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:32:23.221110 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:32:23.221110 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:32:23.221110 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:32:23.221110 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:32:23.222569 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:32:23.222569 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 08 02:32:23.222569 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 08 02:32:23.222569 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 08 02:32:23.222569 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 08 02:32:23.222569 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 08 02:32:23.222569 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 08 02:32:23.222569 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 08 02:32:23.222569 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:32:23.222569 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance cc2b42f1-3dd0-4090-a905-93283047aea3 could not be found. Aug 08 02:32:23.222569 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:23.222569 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:32:23.222569 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:23.222569 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:23.222569 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:32:23.222569 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:32:23.222569 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:32:23.222569 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:32:23.222569 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance cc2b42f1-3dd0-4090-a905-93283047aea3 could not be found. Aug 08 02:32:23.222569 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:23.224041 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-b4401e42-439c-4744-8ed4-0d7b558f1ab7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Returning 404 to user: Instance cc2b42f1-3dd0-4090-a905-93283047aea3 could not be found. {{(pid=118352) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:32:23.224041 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-b4401e42-439c-4744-8ed4-0d7b558f1ab7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "DELETE /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3" status: 404 len: 111 microversion: 2.1 time: 0.030289 Aug 08 02:32:23.224041 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 440/876] 10.222.0.27 () {58 vars in 1239 bytes} [Mon Aug 8 02:32:23 2022] DELETE /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Aug 08 02:32:23.228894 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-4385d4f7-2398-4d71-85bb-3f4cb76825e7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:23.229699 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-4385d4f7-2398-4d71-85bb-3f4cb76825e7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: cc2b42f1-3dd0-4090-a905-93283047aea3] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:23.236378 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4385d4f7-2398-4d71-85bb-3f4cb76825e7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:23.236899 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4385d4f7-2398-4d71-85bb-3f4cb76825e7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:23.237335 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4385d4f7-2398-4d71-85bb-3f4cb76825e7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:23.238026 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4385d4f7-2398-4d71-85bb-3f4cb76825e7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:23.239323 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4385d4f7-2398-4d71-85bb-3f4cb76825e7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:23.239323 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4385d4f7-2398-4d71-85bb-3f4cb76825e7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:23.253542 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.wsgi [None req-4385d4f7-2398-4d71-85bb-3f4cb76825e7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] HTTP exception thrown: Instance cc2b42f1-3dd0-4090-a905-93283047aea3 could not be found. Aug 08 02:32:23.255002 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-4385d4f7-2398-4d71-85bb-3f4cb76825e7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Request method failure captured: Aug 08 02:32:23.255002 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: request: GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3 HTTP/1.1 Aug 08 02:32:23.255002 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Accept: application/json Aug 08 02:32:23.255002 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Accept-Encoding: identity Aug 08 02:32:23.255002 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Connection: close Aug 08 02:32:23.255002 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Content-Length: 0 Aug 08 02:32:23.255002 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Content-Type: application/json Aug 08 02:32:23.255002 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Host: 10.222.0.27 Aug 08 02:32:23.255002 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Openstack-System-Scope: None Aug 08 02:32:23.255002 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: User-Agent: python-urllib3/1.26.11 Aug 08 02:32:23.255002 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Auth-Token: gAAAAABi8HQUs7QI_wjv0vSDVlijMnZ9-Lkgqc--Bu80jwWL-CoTBjbpe1sae_INrRIGeFqgwK-ezXnEOgqpG2NO5P97SEgPcY4zY_QgnaTA6xv-rzef5kDF5s4pU5M8QdM-T8eUF020_v3zsmgR78rxXaglg0C5QuxYvT8R4THTJZuwa_f26Bo Aug 08 02:32:23.255002 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Domain-Id: None Aug 08 02:32:23.255002 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Domain-Name: None Aug 08 02:32:23.255002 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Identity-Status: Confirmed Aug 08 02:32:23.255002 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Is-Admin-Project: True Aug 08 02:32:23.255002 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Domain-Id: default Aug 08 02:32:23.255002 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Domain-Name: Default Aug 08 02:32:23.255002 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Id: ca29b2b9a59d4826bfd25f4dcfa9cffe Aug 08 02:32:23.255002 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Name: tempest-AttachVolumeNegativeTest-420450256 Aug 08 02:32:23.255002 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Role: reader,member Aug 08 02:32:23.255002 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Roles: reader,member Aug 08 02:32:23.257335 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/ca29b2b9a59d4826bfd25f4dcfa9cffe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/ca29b2b9a59d4826bfd25f4dcfa9cffe"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_ca29b2b9a59d4826bfd25f4dcfa9cffe", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/ca29b2b9a59d4826bfd25f4dcfa9cffe"}]}] Aug 08 02:32:23.257335 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant: tempest-AttachVolumeNegativeTest-420450256 Aug 08 02:32:23.257335 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant-Id: ca29b2b9a59d4826bfd25f4dcfa9cffe Aug 08 02:32:23.257335 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-420450256 Aug 08 02:32:23.257335 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User: tempest-AttachVolumeNegativeTest-420450256-project Aug 08 02:32:23.257335 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Domain-Id: default Aug 08 02:32:23.257335 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Domain-Name: Default Aug 08 02:32:23.257335 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Id: 578cd837d9fe442587f8c6c8528334c0 Aug 08 02:32:23.257335 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Name: tempest-AttachVolumeNegativeTest-420450256-project Aug 08 02:32:23.257335 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: method: > Aug 08 02:32:23.257335 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: exception: Instance cc2b42f1-3dd0-4090-a905-93283047aea3 could not be found. Aug 08 02:32:23.257335 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: action_args: {'id': 'cc2b42f1-3dd0-4090-a905-93283047aea3'} Aug 08 02:32:23.258121 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:32:23.258121 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:23.258121 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 08 02:32:23.258121 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 08 02:32:23.258121 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 08 02:32:23.258121 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 08 02:32:23.258121 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 08 02:32:23.258121 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 08 02:32:23.258121 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 08 02:32:23.258121 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 08 02:32:23.258121 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance cc2b42f1-3dd0-4090-a905-93283047aea3 could not be found. Aug 08 02:32:23.258121 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:32:23.258121 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:32:23.258121 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:32:23.258121 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:23.258121 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:32:23.258121 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:32:23.258121 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:32:23.258121 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:32:23.258121 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:32:23.259352 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:32:23.259352 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 08 02:32:23.259352 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 08 02:32:23.259352 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 08 02:32:23.259352 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 08 02:32:23.259352 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 08 02:32:23.259352 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:32:23.259352 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance cc2b42f1-3dd0-4090-a905-93283047aea3 could not be found. Aug 08 02:32:23.259352 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:32:23.259352 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:32:23.259352 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:32:23.259352 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:23.259352 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:32:23.259352 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:32:23.259352 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:32:23.259352 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:32:23.259352 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance cc2b42f1-3dd0-4090-a905-93283047aea3 could not be found. Aug 08 02:32:23.259352 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:32:23.260463 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-4385d4f7-2398-4d71-85bb-3f4cb76825e7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Returning 404 to user: Instance cc2b42f1-3dd0-4090-a905-93283047aea3 could not be found. {{(pid=118351) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:32:23.260463 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-4385d4f7-2398-4d71-85bb-3f4cb76825e7 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3" status: 404 len: 111 microversion: 2.1 time: 0.031341 Aug 08 02:32:23.260463 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 437/877] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:23 2022] GET /compute/v2.1/servers/cc2b42f1-3dd0-4090-a905-93283047aea3 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 08 02:32:24.207558 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-039f45b5-b00e-4a1a-befb-4d787ae78c75 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:24.208199 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-039f45b5-b00e-4a1a-befb-4d787ae78c75 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:24.215514 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-039f45b5-b00e-4a1a-befb-4d787ae78c75 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:24.216007 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-039f45b5-b00e-4a1a-befb-4d787ae78c75 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:24.216481 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-039f45b5-b00e-4a1a-befb-4d787ae78c75 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:24.217253 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-039f45b5-b00e-4a1a-befb-4d787ae78c75 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:24.217714 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-039f45b5-b00e-4a1a-befb-4d787ae78c75 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:24.218151 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-039f45b5-b00e-4a1a-befb-4d787ae78c75 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:24.397650 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-039f45b5-b00e-4a1a-befb-4d787ae78c75 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:24.406258 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-039f45b5-b00e-4a1a-befb-4d787ae78c75 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1568 microversion: 2.1 time: 0.202731 Aug 08 02:32:24.406820 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 441/878] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:24 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1568 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:32:24.428160 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-1af85158-89ee-4ff4-bacc-ef5c1c4bfbd7 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:32:24.428794 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-1af85158-89ee-4ff4-bacc-ef5c1c4bfbd7 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:24.436182 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1af85158-89ee-4ff4-bacc-ef5c1c4bfbd7 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:24.436711 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1af85158-89ee-4ff4-bacc-ef5c1c4bfbd7 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:24.437149 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1af85158-89ee-4ff4-bacc-ef5c1c4bfbd7 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:24.437845 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1af85158-89ee-4ff4-bacc-ef5c1c4bfbd7 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:24.438293 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1af85158-89ee-4ff4-bacc-ef5c1c4bfbd7 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:24.438718 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1af85158-89ee-4ff4-bacc-ef5c1c4bfbd7 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:24.469541 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-1af85158-89ee-4ff4-bacc-ef5c1c4bfbd7 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=118351) unshelve /opt/stack/nova/nova/compute/api.py:4531}} Aug 08 02:32:24.612956 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-1b2aa3c8-8e34-4c15-b0cf-336937c52886 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:24.613582 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-1b2aa3c8-8e34-4c15-b0cf-336937c52886 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: aa20d25c-bd00-41e6-b350-309368839578] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:24.620132 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1b2aa3c8-8e34-4c15-b0cf-336937c52886 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:24.628084 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1b2aa3c8-8e34-4c15-b0cf-336937c52886 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:24.628084 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1b2aa3c8-8e34-4c15-b0cf-336937c52886 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:24.628084 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1b2aa3c8-8e34-4c15-b0cf-336937c52886 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:24.628084 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1b2aa3c8-8e34-4c15-b0cf-336937c52886 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:24.628589 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1b2aa3c8-8e34-4c15-b0cf-336937c52886 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:24.639185 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.wsgi [None req-1b2aa3c8-8e34-4c15-b0cf-336937c52886 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] HTTP exception thrown: Instance aa20d25c-bd00-41e6-b350-309368839578 could not be found. Aug 08 02:32:24.641294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-1b2aa3c8-8e34-4c15-b0cf-336937c52886 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Request method failure captured: Aug 08 02:32:24.641294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: request: DELETE /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578 HTTP/1.1 Aug 08 02:32:24.641294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept: application/json Aug 08 02:32:24.641294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept-Encoding: identity Aug 08 02:32:24.641294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Connection: close Aug 08 02:32:24.641294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Length: 0 Aug 08 02:32:24.641294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Type: application/json Aug 08 02:32:24.641294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Host: 10.222.0.27 Aug 08 02:32:24.641294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Openstack-System-Scope: None Aug 08 02:32:24.641294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: User-Agent: python-urllib3/1.26.11 Aug 08 02:32:24.641294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Auth-Token: gAAAAABi8HQUs7QI_wjv0vSDVlijMnZ9-Lkgqc--Bu80jwWL-CoTBjbpe1sae_INrRIGeFqgwK-ezXnEOgqpG2NO5P97SEgPcY4zY_QgnaTA6xv-rzef5kDF5s4pU5M8QdM-T8eUF020_v3zsmgR78rxXaglg0C5QuxYvT8R4THTJZuwa_f26Bo Aug 08 02:32:24.641294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Id: None Aug 08 02:32:24.641294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Name: None Aug 08 02:32:24.641294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Identity-Status: Confirmed Aug 08 02:32:24.641294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Is-Admin-Project: True Aug 08 02:32:24.641294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Id: default Aug 08 02:32:24.641294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Name: Default Aug 08 02:32:24.641294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Id: ca29b2b9a59d4826bfd25f4dcfa9cffe Aug 08 02:32:24.641294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Name: tempest-AttachVolumeNegativeTest-420450256 Aug 08 02:32:24.641294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Role: reader,member Aug 08 02:32:24.641294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Roles: reader,member Aug 08 02:32:24.642806 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/ca29b2b9a59d4826bfd25f4dcfa9cffe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/ca29b2b9a59d4826bfd25f4dcfa9cffe"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_ca29b2b9a59d4826bfd25f4dcfa9cffe", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/ca29b2b9a59d4826bfd25f4dcfa9cffe"}]}] Aug 08 02:32:24.642806 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant: tempest-AttachVolumeNegativeTest-420450256 Aug 08 02:32:24.642806 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Id: ca29b2b9a59d4826bfd25f4dcfa9cffe Aug 08 02:32:24.642806 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-420450256 Aug 08 02:32:24.642806 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User: tempest-AttachVolumeNegativeTest-420450256-project Aug 08 02:32:24.642806 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Id: default Aug 08 02:32:24.642806 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Name: Default Aug 08 02:32:24.642806 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Id: 578cd837d9fe442587f8c6c8528334c0 Aug 08 02:32:24.642806 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Name: tempest-AttachVolumeNegativeTest-420450256-project Aug 08 02:32:24.642806 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: method: > Aug 08 02:32:24.642806 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: exception: Instance aa20d25c-bd00-41e6-b350-309368839578 could not be found. Aug 08 02:32:24.642806 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: action_args: {'id': 'aa20d25c-bd00-41e6-b350-309368839578'} Aug 08 02:32:24.643576 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:32:24.643576 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:24.643576 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 08 02:32:24.643576 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 08 02:32:24.643576 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 08 02:32:24.643576 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 08 02:32:24.643576 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 08 02:32:24.643576 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 08 02:32:24.643576 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 08 02:32:24.643576 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 08 02:32:24.643576 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance aa20d25c-bd00-41e6-b350-309368839578 could not be found. Aug 08 02:32:24.643576 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:24.643576 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:32:24.643576 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:24.643576 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:24.643576 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:32:24.643576 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:32:24.643576 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:32:24.643576 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:32:24.643576 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:32:24.645160 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:32:24.645160 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 08 02:32:24.645160 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 08 02:32:24.645160 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 08 02:32:24.645160 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 08 02:32:24.645160 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 08 02:32:24.645160 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 08 02:32:24.645160 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 08 02:32:24.645160 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:32:24.645160 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance aa20d25c-bd00-41e6-b350-309368839578 could not be found. Aug 08 02:32:24.645160 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:24.645160 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:32:24.645160 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:24.645160 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:24.645160 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:32:24.645160 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:32:24.645160 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:32:24.645160 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:32:24.645160 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance aa20d25c-bd00-41e6-b350-309368839578 could not be found. Aug 08 02:32:24.645160 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:24.647626 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-1b2aa3c8-8e34-4c15-b0cf-336937c52886 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Returning 404 to user: Instance aa20d25c-bd00-41e6-b350-309368839578 could not be found. {{(pid=118352) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:32:24.648540 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-1b2aa3c8-8e34-4c15-b0cf-336937c52886 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "DELETE /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578" status: 404 len: 111 microversion: 2.1 time: 0.039989 Aug 08 02:32:24.649145 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 442/879] 10.222.0.27 () {58 vars in 1239 bytes} [Mon Aug 8 02:32:24 2022] DELETE /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578 => generated 111 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Aug 08 02:32:24.658389 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-acd7d894-81f6-452a-80a8-880a1912314b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:24.659068 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-acd7d894-81f6-452a-80a8-880a1912314b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: aa20d25c-bd00-41e6-b350-309368839578] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:24.665604 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-acd7d894-81f6-452a-80a8-880a1912314b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:24.666098 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-acd7d894-81f6-452a-80a8-880a1912314b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:24.666571 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-acd7d894-81f6-452a-80a8-880a1912314b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:24.667280 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-acd7d894-81f6-452a-80a8-880a1912314b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:24.667765 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-acd7d894-81f6-452a-80a8-880a1912314b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:24.668264 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-acd7d894-81f6-452a-80a8-880a1912314b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:24.682925 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.wsgi [None req-acd7d894-81f6-452a-80a8-880a1912314b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] HTTP exception thrown: Instance aa20d25c-bd00-41e6-b350-309368839578 could not be found. Aug 08 02:32:24.684507 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-acd7d894-81f6-452a-80a8-880a1912314b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Request method failure captured: Aug 08 02:32:24.684507 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: request: GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578 HTTP/1.1 Aug 08 02:32:24.684507 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept: application/json Aug 08 02:32:24.684507 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept-Encoding: identity Aug 08 02:32:24.684507 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Connection: close Aug 08 02:32:24.684507 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Length: 0 Aug 08 02:32:24.684507 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Type: application/json Aug 08 02:32:24.684507 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Host: 10.222.0.27 Aug 08 02:32:24.684507 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Openstack-System-Scope: None Aug 08 02:32:24.684507 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: User-Agent: python-urllib3/1.26.11 Aug 08 02:32:24.684507 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Auth-Token: gAAAAABi8HQUs7QI_wjv0vSDVlijMnZ9-Lkgqc--Bu80jwWL-CoTBjbpe1sae_INrRIGeFqgwK-ezXnEOgqpG2NO5P97SEgPcY4zY_QgnaTA6xv-rzef5kDF5s4pU5M8QdM-T8eUF020_v3zsmgR78rxXaglg0C5QuxYvT8R4THTJZuwa_f26Bo Aug 08 02:32:24.684507 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Id: None Aug 08 02:32:24.684507 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Name: None Aug 08 02:32:24.684507 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Identity-Status: Confirmed Aug 08 02:32:24.684507 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Is-Admin-Project: True Aug 08 02:32:24.684507 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Id: default Aug 08 02:32:24.684507 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Name: Default Aug 08 02:32:24.684507 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Id: ca29b2b9a59d4826bfd25f4dcfa9cffe Aug 08 02:32:24.684507 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Name: tempest-AttachVolumeNegativeTest-420450256 Aug 08 02:32:24.684507 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Role: reader,member Aug 08 02:32:24.684507 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Roles: reader,member Aug 08 02:32:24.686248 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/ca29b2b9a59d4826bfd25f4dcfa9cffe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/ca29b2b9a59d4826bfd25f4dcfa9cffe"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_ca29b2b9a59d4826bfd25f4dcfa9cffe", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/ca29b2b9a59d4826bfd25f4dcfa9cffe"}]}] Aug 08 02:32:24.686248 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant: tempest-AttachVolumeNegativeTest-420450256 Aug 08 02:32:24.686248 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Id: ca29b2b9a59d4826bfd25f4dcfa9cffe Aug 08 02:32:24.686248 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-420450256 Aug 08 02:32:24.686248 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User: tempest-AttachVolumeNegativeTest-420450256-project Aug 08 02:32:24.686248 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Id: default Aug 08 02:32:24.686248 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Name: Default Aug 08 02:32:24.686248 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Id: 578cd837d9fe442587f8c6c8528334c0 Aug 08 02:32:24.686248 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Name: tempest-AttachVolumeNegativeTest-420450256-project Aug 08 02:32:24.686248 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: method: > Aug 08 02:32:24.686248 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: exception: Instance aa20d25c-bd00-41e6-b350-309368839578 could not be found. Aug 08 02:32:24.686248 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: action_args: {'id': 'aa20d25c-bd00-41e6-b350-309368839578'} Aug 08 02:32:24.687463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:32:24.687463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:24.687463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 08 02:32:24.687463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 08 02:32:24.687463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 08 02:32:24.687463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 08 02:32:24.687463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 08 02:32:24.687463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 08 02:32:24.687463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 08 02:32:24.687463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 08 02:32:24.687463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance aa20d25c-bd00-41e6-b350-309368839578 could not be found. Aug 08 02:32:24.687463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:24.687463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:32:24.687463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:24.687463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:24.687463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:32:24.687463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:32:24.687463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:32:24.687463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:32:24.687463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:32:24.689290 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:32:24.689290 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 08 02:32:24.689290 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 08 02:32:24.689290 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 08 02:32:24.689290 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 08 02:32:24.689290 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 08 02:32:24.689290 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:32:24.689290 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance aa20d25c-bd00-41e6-b350-309368839578 could not be found. Aug 08 02:32:24.689290 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:24.689290 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:32:24.689290 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:24.689290 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:24.689290 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:32:24.689290 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:32:24.689290 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:32:24.689290 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:32:24.689290 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance aa20d25c-bd00-41e6-b350-309368839578 could not be found. Aug 08 02:32:24.689290 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:24.691385 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-acd7d894-81f6-452a-80a8-880a1912314b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Returning 404 to user: Instance aa20d25c-bd00-41e6-b350-309368839578 could not be found. {{(pid=118352) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:32:24.691385 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-acd7d894-81f6-452a-80a8-880a1912314b tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578" status: 404 len: 111 microversion: 2.1 time: 0.031700 Aug 08 02:32:24.691385 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 443/880] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:24 2022] GET /compute/v2.1/servers/aa20d25c-bd00-41e6-b350-309368839578 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Aug 08 02:32:24.694666 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-7d8720ec-3ec2-44d7-b9d2-e06f83d802b2 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:24.696774 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-7d8720ec-3ec2-44d7-b9d2-e06f83d802b2 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: dda0c0b0-ed8e-4d43-9b26-abe1bca05a61] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:24.702280 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7d8720ec-3ec2-44d7-b9d2-e06f83d802b2 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:24.702834 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7d8720ec-3ec2-44d7-b9d2-e06f83d802b2 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:24.703343 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7d8720ec-3ec2-44d7-b9d2-e06f83d802b2 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:24.704052 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7d8720ec-3ec2-44d7-b9d2-e06f83d802b2 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:24.704511 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7d8720ec-3ec2-44d7-b9d2-e06f83d802b2 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:24.704980 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7d8720ec-3ec2-44d7-b9d2-e06f83d802b2 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:24.721032 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.wsgi [None req-7d8720ec-3ec2-44d7-b9d2-e06f83d802b2 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] HTTP exception thrown: Instance dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 could not be found. Aug 08 02:32:24.722603 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-7d8720ec-3ec2-44d7-b9d2-e06f83d802b2 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Request method failure captured: Aug 08 02:32:24.722603 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: request: DELETE /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 HTTP/1.1 Aug 08 02:32:24.722603 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept: application/json Aug 08 02:32:24.722603 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept-Encoding: identity Aug 08 02:32:24.722603 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Connection: close Aug 08 02:32:24.722603 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Length: 0 Aug 08 02:32:24.722603 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Type: application/json Aug 08 02:32:24.722603 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Host: 10.222.0.27 Aug 08 02:32:24.722603 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Openstack-System-Scope: None Aug 08 02:32:24.722603 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: User-Agent: python-urllib3/1.26.11 Aug 08 02:32:24.722603 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Auth-Token: gAAAAABi8HQUs7QI_wjv0vSDVlijMnZ9-Lkgqc--Bu80jwWL-CoTBjbpe1sae_INrRIGeFqgwK-ezXnEOgqpG2NO5P97SEgPcY4zY_QgnaTA6xv-rzef5kDF5s4pU5M8QdM-T8eUF020_v3zsmgR78rxXaglg0C5QuxYvT8R4THTJZuwa_f26Bo Aug 08 02:32:24.722603 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Id: None Aug 08 02:32:24.722603 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Name: None Aug 08 02:32:24.722603 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Identity-Status: Confirmed Aug 08 02:32:24.722603 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Is-Admin-Project: True Aug 08 02:32:24.722603 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Id: default Aug 08 02:32:24.722603 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Name: Default Aug 08 02:32:24.722603 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Id: ca29b2b9a59d4826bfd25f4dcfa9cffe Aug 08 02:32:24.722603 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Name: tempest-AttachVolumeNegativeTest-420450256 Aug 08 02:32:24.722603 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Role: reader,member Aug 08 02:32:24.722603 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Roles: reader,member Aug 08 02:32:24.724273 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/ca29b2b9a59d4826bfd25f4dcfa9cffe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/ca29b2b9a59d4826bfd25f4dcfa9cffe"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_ca29b2b9a59d4826bfd25f4dcfa9cffe", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/ca29b2b9a59d4826bfd25f4dcfa9cffe"}]}] Aug 08 02:32:24.724273 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant: tempest-AttachVolumeNegativeTest-420450256 Aug 08 02:32:24.724273 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Id: ca29b2b9a59d4826bfd25f4dcfa9cffe Aug 08 02:32:24.724273 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-420450256 Aug 08 02:32:24.724273 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User: tempest-AttachVolumeNegativeTest-420450256-project Aug 08 02:32:24.724273 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Id: default Aug 08 02:32:24.724273 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Name: Default Aug 08 02:32:24.724273 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Id: 578cd837d9fe442587f8c6c8528334c0 Aug 08 02:32:24.724273 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Name: tempest-AttachVolumeNegativeTest-420450256-project Aug 08 02:32:24.724273 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: method: > Aug 08 02:32:24.724273 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: exception: Instance dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 could not be found. Aug 08 02:32:24.724273 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: action_args: {'id': 'dda0c0b0-ed8e-4d43-9b26-abe1bca05a61'} Aug 08 02:32:24.725345 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:32:24.725345 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:24.725345 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 08 02:32:24.725345 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 08 02:32:24.725345 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 08 02:32:24.725345 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 08 02:32:24.725345 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 08 02:32:24.725345 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 08 02:32:24.725345 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 08 02:32:24.725345 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 08 02:32:24.725345 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 could not be found. Aug 08 02:32:24.725345 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:24.725345 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:32:24.725345 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:24.725345 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:24.725345 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:32:24.725345 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:32:24.725345 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:32:24.725345 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:32:24.725345 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:32:24.726562 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:32:24.726562 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 08 02:32:24.726562 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 08 02:32:24.726562 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 08 02:32:24.726562 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 08 02:32:24.726562 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 08 02:32:24.726562 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 08 02:32:24.726562 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 08 02:32:24.726562 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:32:24.726562 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 could not be found. Aug 08 02:32:24.726562 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:24.726562 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:32:24.726562 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:24.726562 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:24.726562 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:32:24.726562 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:32:24.726562 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:32:24.726562 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:32:24.726562 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 could not be found. Aug 08 02:32:24.726562 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:24.727867 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-7d8720ec-3ec2-44d7-b9d2-e06f83d802b2 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Returning 404 to user: Instance dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 could not be found. {{(pid=118352) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:32:24.727867 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-7d8720ec-3ec2-44d7-b9d2-e06f83d802b2 tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "DELETE /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61" status: 404 len: 111 microversion: 2.1 time: 0.033256 Aug 08 02:32:24.727867 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 444/881] 10.222.0.27 () {58 vars in 1239 bytes} [Mon Aug 8 02:32:24 2022] DELETE /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 08 02:32:24.733171 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-555c67a0-e040-4aa0-bd0d-16d63bf70b3c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:24.733779 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-555c67a0-e040-4aa0-bd0d-16d63bf70b3c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: dda0c0b0-ed8e-4d43-9b26-abe1bca05a61] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:24.739721 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-555c67a0-e040-4aa0-bd0d-16d63bf70b3c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:24.740197 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-555c67a0-e040-4aa0-bd0d-16d63bf70b3c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:24.740632 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-555c67a0-e040-4aa0-bd0d-16d63bf70b3c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:24.741362 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-555c67a0-e040-4aa0-bd0d-16d63bf70b3c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:24.741812 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-555c67a0-e040-4aa0-bd0d-16d63bf70b3c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:24.742234 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-555c67a0-e040-4aa0-bd0d-16d63bf70b3c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:24.757142 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.wsgi [None req-555c67a0-e040-4aa0-bd0d-16d63bf70b3c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] HTTP exception thrown: Instance dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 could not be found. Aug 08 02:32:24.758976 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-555c67a0-e040-4aa0-bd0d-16d63bf70b3c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Request method failure captured: Aug 08 02:32:24.758976 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: request: GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 HTTP/1.1 Aug 08 02:32:24.758976 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept: application/json Aug 08 02:32:24.758976 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept-Encoding: identity Aug 08 02:32:24.758976 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Connection: close Aug 08 02:32:24.758976 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Length: 0 Aug 08 02:32:24.758976 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Type: application/json Aug 08 02:32:24.758976 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Host: 10.222.0.27 Aug 08 02:32:24.758976 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Openstack-System-Scope: None Aug 08 02:32:24.758976 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: User-Agent: python-urllib3/1.26.11 Aug 08 02:32:24.758976 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Auth-Token: gAAAAABi8HQUs7QI_wjv0vSDVlijMnZ9-Lkgqc--Bu80jwWL-CoTBjbpe1sae_INrRIGeFqgwK-ezXnEOgqpG2NO5P97SEgPcY4zY_QgnaTA6xv-rzef5kDF5s4pU5M8QdM-T8eUF020_v3zsmgR78rxXaglg0C5QuxYvT8R4THTJZuwa_f26Bo Aug 08 02:32:24.758976 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Id: None Aug 08 02:32:24.758976 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Name: None Aug 08 02:32:24.758976 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Identity-Status: Confirmed Aug 08 02:32:24.758976 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Is-Admin-Project: True Aug 08 02:32:24.758976 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Id: default Aug 08 02:32:24.758976 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Name: Default Aug 08 02:32:24.758976 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Id: ca29b2b9a59d4826bfd25f4dcfa9cffe Aug 08 02:32:24.758976 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Name: tempest-AttachVolumeNegativeTest-420450256 Aug 08 02:32:24.758976 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Role: reader,member Aug 08 02:32:24.758976 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Roles: reader,member Aug 08 02:32:24.760617 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/ca29b2b9a59d4826bfd25f4dcfa9cffe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/ca29b2b9a59d4826bfd25f4dcfa9cffe"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_ca29b2b9a59d4826bfd25f4dcfa9cffe", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/ca29b2b9a59d4826bfd25f4dcfa9cffe"}]}] Aug 08 02:32:24.760617 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant: tempest-AttachVolumeNegativeTest-420450256 Aug 08 02:32:24.760617 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Id: ca29b2b9a59d4826bfd25f4dcfa9cffe Aug 08 02:32:24.760617 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-420450256 Aug 08 02:32:24.760617 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User: tempest-AttachVolumeNegativeTest-420450256-project Aug 08 02:32:24.760617 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Id: default Aug 08 02:32:24.760617 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Name: Default Aug 08 02:32:24.760617 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Id: 578cd837d9fe442587f8c6c8528334c0 Aug 08 02:32:24.760617 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Name: tempest-AttachVolumeNegativeTest-420450256-project Aug 08 02:32:24.760617 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: method: > Aug 08 02:32:24.760617 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: exception: Instance dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 could not be found. Aug 08 02:32:24.760617 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: action_args: {'id': 'dda0c0b0-ed8e-4d43-9b26-abe1bca05a61'} Aug 08 02:32:24.762347 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:32:24.762347 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:24.762347 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 08 02:32:24.762347 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 08 02:32:24.762347 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 08 02:32:24.762347 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 08 02:32:24.762347 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 08 02:32:24.762347 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 08 02:32:24.762347 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 08 02:32:24.762347 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 08 02:32:24.762347 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 could not be found. Aug 08 02:32:24.762347 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:24.762347 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:32:24.762347 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:24.762347 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:24.762347 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:32:24.762347 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:32:24.762347 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:32:24.762347 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:32:24.762347 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:32:24.763594 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:32:24.763594 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 08 02:32:24.763594 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 08 02:32:24.763594 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 08 02:32:24.763594 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 08 02:32:24.763594 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 08 02:32:24.763594 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:32:24.763594 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 could not be found. Aug 08 02:32:24.763594 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:24.763594 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:32:24.763594 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:24.763594 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:24.763594 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:32:24.763594 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:32:24.763594 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:32:24.763594 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:32:24.763594 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 could not be found. Aug 08 02:32:24.763594 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:24.768105 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-555c67a0-e040-4aa0-bd0d-16d63bf70b3c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Returning 404 to user: Instance dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 could not be found. {{(pid=118352) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:32:24.768105 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-555c67a0-e040-4aa0-bd0d-16d63bf70b3c tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61" status: 404 len: 111 microversion: 2.1 time: 0.031804 Aug 08 02:32:24.768105 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 445/882] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:24 2022] GET /compute/v2.1/servers/dda0c0b0-ed8e-4d43-9b26-abe1bca05a61 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Aug 08 02:32:24.839133 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-1af85158-89ee-4ff4-bacc-ef5c1c4bfbd7 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "POST /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c/action" status: 202 len: 0 microversion: 2.1 time: 0.414748 Aug 08 02:32:24.840493 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 438/883] 10.222.0.27 () {60 vars in 1278 bytes} [Mon Aug 8 02:32:24 2022] POST /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c/action => generated 0 bytes in 417 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 08 02:32:24.848934 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-3a311021-2872-4bf1-84ac-5cd62583b0c9 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:24.849551 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-3a311021-2872-4bf1-84ac-5cd62583b0c9 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:24.856018 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3a311021-2872-4bf1-84ac-5cd62583b0c9 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:24.856562 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3a311021-2872-4bf1-84ac-5cd62583b0c9 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:24.857242 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3a311021-2872-4bf1-84ac-5cd62583b0c9 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:24.857962 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3a311021-2872-4bf1-84ac-5cd62583b0c9 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:24.858399 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3a311021-2872-4bf1-84ac-5cd62583b0c9 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:24.858825 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3a311021-2872-4bf1-84ac-5cd62583b0c9 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:25.010121 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-3a311021-2872-4bf1-84ac-5cd62583b0c9 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:25.020694 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-3a311021-2872-4bf1-84ac-5cd62583b0c9 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1516 microversion: 2.1 time: 0.175795 Aug 08 02:32:25.021294 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 446/884] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:24 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1516 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:32:26.043661 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-2c0eaa38-0dfa-4cac-9262-01b507617000 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:26.044274 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-2c0eaa38-0dfa-4cac-9262-01b507617000 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:26.050492 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2c0eaa38-0dfa-4cac-9262-01b507617000 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:26.050980 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2c0eaa38-0dfa-4cac-9262-01b507617000 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:26.051425 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2c0eaa38-0dfa-4cac-9262-01b507617000 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:26.052118 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2c0eaa38-0dfa-4cac-9262-01b507617000 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:26.052577 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2c0eaa38-0dfa-4cac-9262-01b507617000 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:26.053038 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2c0eaa38-0dfa-4cac-9262-01b507617000 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:26.138335 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-cc368b2d-2b47-4ef7-a868-2777aec09a9f tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:26.138842 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-cc368b2d-2b47-4ef7-a868-2777aec09a9f tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: 95ae6b45-35d7-43a7-a021-91bb545cac4e] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:26.145619 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cc368b2d-2b47-4ef7-a868-2777aec09a9f tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:26.146110 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cc368b2d-2b47-4ef7-a868-2777aec09a9f tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:26.146548 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cc368b2d-2b47-4ef7-a868-2777aec09a9f tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:26.147245 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cc368b2d-2b47-4ef7-a868-2777aec09a9f tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:26.147697 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cc368b2d-2b47-4ef7-a868-2777aec09a9f tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:26.148124 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cc368b2d-2b47-4ef7-a868-2777aec09a9f tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:26.166135 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.wsgi [None req-cc368b2d-2b47-4ef7-a868-2777aec09a9f tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] HTTP exception thrown: Instance 95ae6b45-35d7-43a7-a021-91bb545cac4e could not be found. Aug 08 02:32:26.167800 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-cc368b2d-2b47-4ef7-a868-2777aec09a9f tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Request method failure captured: Aug 08 02:32:26.167800 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: request: DELETE /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e HTTP/1.1 Aug 08 02:32:26.167800 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept: application/json Aug 08 02:32:26.167800 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept-Encoding: identity Aug 08 02:32:26.167800 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Connection: close Aug 08 02:32:26.167800 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Length: 0 Aug 08 02:32:26.167800 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Type: application/json Aug 08 02:32:26.167800 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Host: 10.222.0.27 Aug 08 02:32:26.167800 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Openstack-System-Scope: None Aug 08 02:32:26.167800 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: User-Agent: python-urllib3/1.26.11 Aug 08 02:32:26.167800 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Auth-Token: gAAAAABi8HQUs7QI_wjv0vSDVlijMnZ9-Lkgqc--Bu80jwWL-CoTBjbpe1sae_INrRIGeFqgwK-ezXnEOgqpG2NO5P97SEgPcY4zY_QgnaTA6xv-rzef5kDF5s4pU5M8QdM-T8eUF020_v3zsmgR78rxXaglg0C5QuxYvT8R4THTJZuwa_f26Bo Aug 08 02:32:26.167800 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Id: None Aug 08 02:32:26.167800 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Name: None Aug 08 02:32:26.167800 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Identity-Status: Confirmed Aug 08 02:32:26.167800 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Is-Admin-Project: True Aug 08 02:32:26.167800 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Id: default Aug 08 02:32:26.167800 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Name: Default Aug 08 02:32:26.167800 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Id: ca29b2b9a59d4826bfd25f4dcfa9cffe Aug 08 02:32:26.167800 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Name: tempest-AttachVolumeNegativeTest-420450256 Aug 08 02:32:26.167800 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Role: reader,member Aug 08 02:32:26.167800 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Roles: reader,member Aug 08 02:32:26.169375 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/ca29b2b9a59d4826bfd25f4dcfa9cffe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/ca29b2b9a59d4826bfd25f4dcfa9cffe"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_ca29b2b9a59d4826bfd25f4dcfa9cffe", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/ca29b2b9a59d4826bfd25f4dcfa9cffe"}]}] Aug 08 02:32:26.169375 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant: tempest-AttachVolumeNegativeTest-420450256 Aug 08 02:32:26.169375 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Id: ca29b2b9a59d4826bfd25f4dcfa9cffe Aug 08 02:32:26.169375 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-420450256 Aug 08 02:32:26.169375 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User: tempest-AttachVolumeNegativeTest-420450256-project Aug 08 02:32:26.169375 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Id: default Aug 08 02:32:26.169375 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Name: Default Aug 08 02:32:26.169375 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Id: 578cd837d9fe442587f8c6c8528334c0 Aug 08 02:32:26.169375 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Name: tempest-AttachVolumeNegativeTest-420450256-project Aug 08 02:32:26.169375 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: method: > Aug 08 02:32:26.169375 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: exception: Instance 95ae6b45-35d7-43a7-a021-91bb545cac4e could not be found. Aug 08 02:32:26.169375 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: action_args: {'id': '95ae6b45-35d7-43a7-a021-91bb545cac4e'} Aug 08 02:32:26.170175 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:32:26.170175 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:26.170175 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 08 02:32:26.170175 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 08 02:32:26.170175 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 08 02:32:26.170175 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 08 02:32:26.170175 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 08 02:32:26.170175 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 08 02:32:26.170175 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 08 02:32:26.170175 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 08 02:32:26.170175 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 95ae6b45-35d7-43a7-a021-91bb545cac4e could not be found. Aug 08 02:32:26.170175 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:26.170175 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:32:26.170175 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:26.170175 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:26.170175 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:32:26.170175 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:32:26.170175 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:32:26.170175 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:32:26.170175 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:32:26.171666 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:32:26.171666 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 08 02:32:26.171666 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 08 02:32:26.171666 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 08 02:32:26.171666 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 08 02:32:26.171666 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 08 02:32:26.171666 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 08 02:32:26.171666 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 08 02:32:26.171666 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:32:26.171666 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 95ae6b45-35d7-43a7-a021-91bb545cac4e could not be found. Aug 08 02:32:26.171666 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:26.171666 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:32:26.171666 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:26.171666 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:26.171666 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:32:26.171666 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:32:26.171666 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:32:26.171666 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:32:26.171666 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 95ae6b45-35d7-43a7-a021-91bb545cac4e could not be found. Aug 08 02:32:26.171666 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:26.173160 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-cc368b2d-2b47-4ef7-a868-2777aec09a9f tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Returning 404 to user: Instance 95ae6b45-35d7-43a7-a021-91bb545cac4e could not be found. {{(pid=118352) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:32:26.173160 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-cc368b2d-2b47-4ef7-a868-2777aec09a9f tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "DELETE /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e" status: 404 len: 111 microversion: 2.1 time: 0.035331 Aug 08 02:32:26.173160 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 447/885] 10.222.0.27 () {58 vars in 1239 bytes} [Mon Aug 8 02:32:26 2022] DELETE /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Aug 08 02:32:26.177869 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-849ec7fb-e158-4474-8754-ceca0f77508e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:26.178576 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-849ec7fb-e158-4474-8754-ceca0f77508e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] [instance: 95ae6b45-35d7-43a7-a021-91bb545cac4e] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:26.184980 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-849ec7fb-e158-4474-8754-ceca0f77508e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:26.185496 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-849ec7fb-e158-4474-8754-ceca0f77508e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:26.186710 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-849ec7fb-e158-4474-8754-ceca0f77508e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:26.186853 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-849ec7fb-e158-4474-8754-ceca0f77508e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:26.187356 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-849ec7fb-e158-4474-8754-ceca0f77508e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:26.187835 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-849ec7fb-e158-4474-8754-ceca0f77508e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:26.201862 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-2c0eaa38-0dfa-4cac-9262-01b507617000 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:26.202365 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.wsgi [None req-849ec7fb-e158-4474-8754-ceca0f77508e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] HTTP exception thrown: Instance 95ae6b45-35d7-43a7-a021-91bb545cac4e could not be found. Aug 08 02:32:26.203908 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-849ec7fb-e158-4474-8754-ceca0f77508e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Request method failure captured: Aug 08 02:32:26.203908 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: request: GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e HTTP/1.1 Aug 08 02:32:26.203908 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept: application/json Aug 08 02:32:26.203908 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept-Encoding: identity Aug 08 02:32:26.203908 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Connection: close Aug 08 02:32:26.203908 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Length: 0 Aug 08 02:32:26.203908 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Type: application/json Aug 08 02:32:26.203908 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Host: 10.222.0.27 Aug 08 02:32:26.203908 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Openstack-System-Scope: None Aug 08 02:32:26.203908 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: User-Agent: python-urllib3/1.26.11 Aug 08 02:32:26.203908 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Auth-Token: gAAAAABi8HQUs7QI_wjv0vSDVlijMnZ9-Lkgqc--Bu80jwWL-CoTBjbpe1sae_INrRIGeFqgwK-ezXnEOgqpG2NO5P97SEgPcY4zY_QgnaTA6xv-rzef5kDF5s4pU5M8QdM-T8eUF020_v3zsmgR78rxXaglg0C5QuxYvT8R4THTJZuwa_f26Bo Aug 08 02:32:26.203908 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Id: None Aug 08 02:32:26.203908 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Name: None Aug 08 02:32:26.203908 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Identity-Status: Confirmed Aug 08 02:32:26.203908 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Is-Admin-Project: True Aug 08 02:32:26.203908 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Id: default Aug 08 02:32:26.203908 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Name: Default Aug 08 02:32:26.203908 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Id: ca29b2b9a59d4826bfd25f4dcfa9cffe Aug 08 02:32:26.203908 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Name: tempest-AttachVolumeNegativeTest-420450256 Aug 08 02:32:26.203908 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Role: reader,member Aug 08 02:32:26.203908 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Roles: reader,member Aug 08 02:32:26.205476 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/ca29b2b9a59d4826bfd25f4dcfa9cffe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/ca29b2b9a59d4826bfd25f4dcfa9cffe"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_ca29b2b9a59d4826bfd25f4dcfa9cffe", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/ca29b2b9a59d4826bfd25f4dcfa9cffe"}]}] Aug 08 02:32:26.205476 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant: tempest-AttachVolumeNegativeTest-420450256 Aug 08 02:32:26.205476 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Id: ca29b2b9a59d4826bfd25f4dcfa9cffe Aug 08 02:32:26.205476 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-420450256 Aug 08 02:32:26.205476 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User: tempest-AttachVolumeNegativeTest-420450256-project Aug 08 02:32:26.205476 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Id: default Aug 08 02:32:26.205476 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Name: Default Aug 08 02:32:26.205476 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Id: 578cd837d9fe442587f8c6c8528334c0 Aug 08 02:32:26.205476 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Name: tempest-AttachVolumeNegativeTest-420450256-project Aug 08 02:32:26.205476 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: method: > Aug 08 02:32:26.205476 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: exception: Instance 95ae6b45-35d7-43a7-a021-91bb545cac4e could not be found. Aug 08 02:32:26.205476 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: action_args: {'id': '95ae6b45-35d7-43a7-a021-91bb545cac4e'} Aug 08 02:32:26.206321 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:32:26.206321 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:26.206321 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 08 02:32:26.206321 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 08 02:32:26.206321 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 08 02:32:26.206321 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 08 02:32:26.206321 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 08 02:32:26.206321 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 08 02:32:26.206321 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 08 02:32:26.206321 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 08 02:32:26.206321 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 95ae6b45-35d7-43a7-a021-91bb545cac4e could not be found. Aug 08 02:32:26.206321 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:26.206321 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:32:26.206321 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:26.206321 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:26.206321 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:32:26.206321 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:32:26.206321 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:32:26.206321 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:32:26.206321 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:32:26.208023 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:32:26.208023 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 08 02:32:26.208023 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 08 02:32:26.208023 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 08 02:32:26.208023 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 08 02:32:26.208023 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 08 02:32:26.208023 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:32:26.208023 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 95ae6b45-35d7-43a7-a021-91bb545cac4e could not be found. Aug 08 02:32:26.208023 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:26.208023 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:32:26.208023 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:26.208023 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:26.208023 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:32:26.208023 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:32:26.208023 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:32:26.208023 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:32:26.208023 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 95ae6b45-35d7-43a7-a021-91bb545cac4e could not be found. Aug 08 02:32:26.208023 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:26.210300 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-849ec7fb-e158-4474-8754-ceca0f77508e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] Returning 404 to user: Instance 95ae6b45-35d7-43a7-a021-91bb545cac4e could not be found. {{(pid=118352) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:32:26.210300 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-849ec7fb-e158-4474-8754-ceca0f77508e tempest-AttachVolumeNegativeTest-420450256 tempest-AttachVolumeNegativeTest-420450256-project] 10.222.0.27 "GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e" status: 404 len: 111 microversion: 2.1 time: 0.032424 Aug 08 02:32:26.210300 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 448/886] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:26 2022] GET /compute/v2.1/servers/95ae6b45-35d7-43a7-a021-91bb545cac4e => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 08 02:32:26.210728 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-2c0eaa38-0dfa-4cac-9262-01b507617000 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1516 microversion: 2.1 time: 0.172020 Aug 08 02:32:26.211159 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 439/887] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:26 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1516 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:32:26.762426 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [req-d9bd381d-ab2f-4dda-85ce-be3b5d3fdbc4 req-fb3b16d3-e712-4419-9d15-877f441730ac service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "077e7935-3361-4ff5-9367-ba162efac09c", "name": "network-vif-unplugged", "status": "completed", "tag": "3051cabc-48a3-4055-ae74-0ae74aa73005"}]} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:32:26.774024 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-d9bd381d-ab2f-4dda-85ce-be3b5d3fdbc4 req-fb3b16d3-e712-4419-9d15-877f441730ac service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:26.774581 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-d9bd381d-ab2f-4dda-85ce-be3b5d3fdbc4 req-fb3b16d3-e712-4419-9d15-877f441730ac service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:26.775076 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-d9bd381d-ab2f-4dda-85ce-be3b5d3fdbc4 req-fb3b16d3-e712-4419-9d15-877f441730ac service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:26.799734 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.compute.server_external_events [req-d9bd381d-ab2f-4dda-85ce-be3b5d3fdbc4 req-fb3b16d3-e712-4419-9d15-877f441730ac service nova] Unable to find a host for instance 077e7935-3361-4ff5-9367-ba162efac09c. Dropping event network-vif-unplugged {{(pid=118352) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:143}} Aug 08 02:32:26.799734 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [req-d9bd381d-ab2f-4dda-85ce-be3b5d3fdbc4 req-fb3b16d3-e712-4419-9d15-877f441730ac service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 207 len: 182 microversion: 2.1 time: 0.037925 Aug 08 02:32:26.799734 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 449/888] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:32:26 2022] POST /compute/v2.1/os-server-external-events => generated 182 bytes in 40 msecs (HTTP/1.1 207) 9 headers in 367 bytes (1 switches on core 0) Aug 08 02:32:27.233308 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-0f8fba5d-1419-402a-8f62-28bf5d4f8b70 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:27.233942 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-0f8fba5d-1419-402a-8f62-28bf5d4f8b70 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:27.240548 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0f8fba5d-1419-402a-8f62-28bf5d4f8b70 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:27.241129 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0f8fba5d-1419-402a-8f62-28bf5d4f8b70 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:27.241553 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0f8fba5d-1419-402a-8f62-28bf5d4f8b70 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:27.242254 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0f8fba5d-1419-402a-8f62-28bf5d4f8b70 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:27.242718 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0f8fba5d-1419-402a-8f62-28bf5d4f8b70 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:27.243328 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0f8fba5d-1419-402a-8f62-28bf5d4f8b70 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:27.461386 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-0f8fba5d-1419-402a-8f62-28bf5d4f8b70 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:27.469542 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-0f8fba5d-1419-402a-8f62-28bf5d4f8b70 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1518 microversion: 2.1 time: 0.240793 Aug 08 02:32:27.470113 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 440/889] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:27 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1518 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:32:28.491384 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-62ad6e9c-fafc-49d1-a918-3dd203854b37 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:28.491970 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-62ad6e9c-fafc-49d1-a918-3dd203854b37 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:28.499178 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-62ad6e9c-fafc-49d1-a918-3dd203854b37 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:28.499671 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-62ad6e9c-fafc-49d1-a918-3dd203854b37 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:28.500565 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-62ad6e9c-fafc-49d1-a918-3dd203854b37 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:28.500876 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-62ad6e9c-fafc-49d1-a918-3dd203854b37 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:28.501333 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-62ad6e9c-fafc-49d1-a918-3dd203854b37 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:28.501755 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-62ad6e9c-fafc-49d1-a918-3dd203854b37 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:28.687526 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-62ad6e9c-fafc-49d1-a918-3dd203854b37 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:28.695534 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-62ad6e9c-fafc-49d1-a918-3dd203854b37 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1574 microversion: 2.1 time: 0.208049 Aug 08 02:32:28.709999 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 450/890] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:28 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1574 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:32:29.717022 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-d0e35bb8-6b54-43a2-96ba-98e50aafdf2f tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:29.717636 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-d0e35bb8-6b54-43a2-96ba-98e50aafdf2f tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:29.724089 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d0e35bb8-6b54-43a2-96ba-98e50aafdf2f tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:29.724569 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d0e35bb8-6b54-43a2-96ba-98e50aafdf2f tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:29.725029 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d0e35bb8-6b54-43a2-96ba-98e50aafdf2f tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:29.725713 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d0e35bb8-6b54-43a2-96ba-98e50aafdf2f tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:29.726161 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d0e35bb8-6b54-43a2-96ba-98e50aafdf2f tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:29.726578 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d0e35bb8-6b54-43a2-96ba-98e50aafdf2f tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:29.877298 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-d0e35bb8-6b54-43a2-96ba-98e50aafdf2f tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:29.897386 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-d0e35bb8-6b54-43a2-96ba-98e50aafdf2f tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1574 microversion: 2.1 time: 0.173091 Aug 08 02:32:29.898069 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 441/891] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:29 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1574 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:32:30.921052 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-d559e0e8-69b1-4f85-a631-64b61f1cf3f9 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:30.922229 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-d559e0e8-69b1-4f85-a631-64b61f1cf3f9 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:30.930920 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d559e0e8-69b1-4f85-a631-64b61f1cf3f9 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:30.933982 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d559e0e8-69b1-4f85-a631-64b61f1cf3f9 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:30.933982 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d559e0e8-69b1-4f85-a631-64b61f1cf3f9 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:30.933982 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d559e0e8-69b1-4f85-a631-64b61f1cf3f9 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:30.933982 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d559e0e8-69b1-4f85-a631-64b61f1cf3f9 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:30.934349 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d559e0e8-69b1-4f85-a631-64b61f1cf3f9 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:31.074087 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-d559e0e8-69b1-4f85-a631-64b61f1cf3f9 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:31.082112 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-d559e0e8-69b1-4f85-a631-64b61f1cf3f9 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1574 microversion: 2.1 time: 0.167030 Aug 08 02:32:31.082689 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 451/892] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:30 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1574 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:32:32.103140 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-05e56241-40df-4de1-b416-08c040c5bcf0 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:32.103834 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-05e56241-40df-4de1-b416-08c040c5bcf0 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:32.110480 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-05e56241-40df-4de1-b416-08c040c5bcf0 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:32.110966 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-05e56241-40df-4de1-b416-08c040c5bcf0 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:32.111406 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-05e56241-40df-4de1-b416-08c040c5bcf0 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:32.112098 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-05e56241-40df-4de1-b416-08c040c5bcf0 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:32.112547 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-05e56241-40df-4de1-b416-08c040c5bcf0 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:32.112999 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-05e56241-40df-4de1-b416-08c040c5bcf0 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:32.249977 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-05e56241-40df-4de1-b416-08c040c5bcf0 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:32.257669 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-05e56241-40df-4de1-b416-08c040c5bcf0 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1574 microversion: 2.1 time: 0.158296 Aug 08 02:32:32.258240 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 442/893] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:32 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1574 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:32:32.818559 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-cd18a8bb-ea41-42e7-a747-c5d32b29cae0 tempest-VolumesGetTestJSON-107376284 tempest-VolumesGetTestJSON-107376284-project] Action: 'create', calling method: .version_select at 0x7efc6f7ed8b0>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesGetTestJSON-Volume-1685351921", "metadata": {"Type": "work"}}} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:32:33.279379 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-b8ed46c2-171a-4da3-b8a0-65c5f29d1e7d tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:33.280083 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-b8ed46c2-171a-4da3-b8a0-65c5f29d1e7d tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:33.286898 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b8ed46c2-171a-4da3-b8a0-65c5f29d1e7d tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:33.287384 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b8ed46c2-171a-4da3-b8a0-65c5f29d1e7d tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:33.288040 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b8ed46c2-171a-4da3-b8a0-65c5f29d1e7d tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:33.288788 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b8ed46c2-171a-4da3-b8a0-65c5f29d1e7d tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:33.289241 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b8ed46c2-171a-4da3-b8a0-65c5f29d1e7d tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:33.290193 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b8ed46c2-171a-4da3-b8a0-65c5f29d1e7d tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:33.581726 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-b8ed46c2-171a-4da3-b8a0-65c5f29d1e7d tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:33.603447 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-b8ed46c2-171a-4da3-b8a0-65c5f29d1e7d tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1574 microversion: 2.1 time: 0.318118 Aug 08 02:32:33.606916 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-cd18a8bb-ea41-42e7-a747-c5d32b29cae0 tempest-VolumesGetTestJSON-107376284 tempest-VolumesGetTestJSON-107376284-project] 10.222.0.27 "POST /compute/v2.1/os-volumes" status: 200 len: 353 microversion: 2.1 time: 0.925817 Aug 08 02:32:33.607748 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 452/894] 10.222.0.27 () {60 vars in 1156 bytes} [Mon Aug 8 02:32:32 2022] POST /compute/v2.1/os-volumes => generated 353 bytes in 927 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Aug 08 02:32:33.608686 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 443/895] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:33 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1574 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:32:33.620066 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-292c83c6-e027-46b5-979c-2fa9c314f2ce tempest-VolumesGetTestJSON-107376284 tempest-VolumesGetTestJSON-107376284-project] Calling method '.version_select at 0x7efc6f7f7160>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:33.671062 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-292c83c6-e027-46b5-979c-2fa9c314f2ce tempest-VolumesGetTestJSON-107376284 tempest-VolumesGetTestJSON-107376284-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/405121b9-a73e-4dbd-8b5d-e5b850fc7dee" status: 200 len: 353 microversion: 2.1 time: 0.054610 Aug 08 02:32:33.671632 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 453/896] 10.222.0.27 () {58 vars in 1245 bytes} [Mon Aug 8 02:32:33 2022] GET /compute/v2.1/os-volumes/405121b9-a73e-4dbd-8b5d-e5b850fc7dee => generated 353 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:32:34.614864 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-3c7962b7-6980-4721-a7f8-dfdb91249b40 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:34.615469 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-3c7962b7-6980-4721-a7f8-dfdb91249b40 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:34.623967 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3c7962b7-6980-4721-a7f8-dfdb91249b40 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:34.624454 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3c7962b7-6980-4721-a7f8-dfdb91249b40 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:34.624914 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3c7962b7-6980-4721-a7f8-dfdb91249b40 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:34.625596 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3c7962b7-6980-4721-a7f8-dfdb91249b40 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:34.626046 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3c7962b7-6980-4721-a7f8-dfdb91249b40 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:34.626462 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3c7962b7-6980-4721-a7f8-dfdb91249b40 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:34.686300 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-3b990eaf-99b5-40c8-b1e9-2249fb2163b5 tempest-VolumesGetTestJSON-107376284 tempest-VolumesGetTestJSON-107376284-project] Calling method '.version_select at 0x7efc740818b0>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:34.735233 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-3b990eaf-99b5-40c8-b1e9-2249fb2163b5 tempest-VolumesGetTestJSON-107376284 tempest-VolumesGetTestJSON-107376284-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/405121b9-a73e-4dbd-8b5d-e5b850fc7dee" status: 200 len: 354 microversion: 2.1 time: 0.054115 Aug 08 02:32:34.735806 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 454/897] 10.222.0.27 () {58 vars in 1245 bytes} [Mon Aug 8 02:32:34 2022] GET /compute/v2.1/os-volumes/405121b9-a73e-4dbd-8b5d-e5b850fc7dee => generated 354 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 08 02:32:34.748712 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-04410ced-778e-4e98-ac6c-f51837402d2f tempest-VolumesGetTestJSON-107376284 tempest-VolumesGetTestJSON-107376284-project] Calling method '.version_select at 0x7efc6f7f75e0>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:34.781942 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-3c7962b7-6980-4721-a7f8-dfdb91249b40 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:34.789755 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-3c7962b7-6980-4721-a7f8-dfdb91249b40 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1574 microversion: 2.1 time: 0.178753 Aug 08 02:32:34.790465 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 444/898] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:34 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1574 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:32:34.796064 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-04410ced-778e-4e98-ac6c-f51837402d2f tempest-VolumesGetTestJSON-107376284 tempest-VolumesGetTestJSON-107376284-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/405121b9-a73e-4dbd-8b5d-e5b850fc7dee" status: 200 len: 354 microversion: 2.1 time: 0.051672 Aug 08 02:32:34.796631 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 455/899] 10.222.0.27 () {58 vars in 1245 bytes} [Mon Aug 8 02:32:34 2022] GET /compute/v2.1/os-volumes/405121b9-a73e-4dbd-8b5d-e5b850fc7dee => generated 354 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 08 02:32:34.814790 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-ad7fe5d5-eb20-4f94-959c-50f0fc5e07ca tempest-VolumesGetTestJSON-107376284 tempest-VolumesGetTestJSON-107376284-project] Calling method '.version_select at 0x7efc6f7dec10>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:35.052422 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-ad7fe5d5-eb20-4f94-959c-50f0fc5e07ca tempest-VolumesGetTestJSON-107376284 tempest-VolumesGetTestJSON-107376284-project] 10.222.0.27 "DELETE /compute/v2.1/os-volumes/405121b9-a73e-4dbd-8b5d-e5b850fc7dee" status: 202 len: 0 microversion: 2.1 time: 0.241045 Aug 08 02:32:35.052682 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 445/900] 10.222.0.27 () {58 vars in 1248 bytes} [Mon Aug 8 02:32:34 2022] DELETE /compute/v2.1/os-volumes/405121b9-a73e-4dbd-8b5d-e5b850fc7dee => generated 0 bytes in 242 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 08 02:32:35.063446 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-b8ce9aec-42ef-4b87-8640-e2380f770421 tempest-VolumesGetTestJSON-107376284 tempest-VolumesGetTestJSON-107376284-project] Calling method '.version_select at 0x7efc6f7ed4c0>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:35.113620 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-b8ce9aec-42ef-4b87-8640-e2380f770421 tempest-VolumesGetTestJSON-107376284 tempest-VolumesGetTestJSON-107376284-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/405121b9-a73e-4dbd-8b5d-e5b850fc7dee" status: 200 len: 353 microversion: 2.1 time: 0.056196 Aug 08 02:32:35.114189 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 456/901] 10.222.0.27 () {58 vars in 1245 bytes} [Mon Aug 8 02:32:35 2022] GET /compute/v2.1/os-volumes/405121b9-a73e-4dbd-8b5d-e5b850fc7dee => generated 353 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:32:35.812853 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-859ca77e-ad25-420d-9301-7e57d7400058 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:35.813485 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-859ca77e-ad25-420d-9301-7e57d7400058 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:35.819892 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-859ca77e-ad25-420d-9301-7e57d7400058 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:35.820407 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-859ca77e-ad25-420d-9301-7e57d7400058 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:35.820888 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-859ca77e-ad25-420d-9301-7e57d7400058 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:35.821862 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-859ca77e-ad25-420d-9301-7e57d7400058 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:35.822341 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-859ca77e-ad25-420d-9301-7e57d7400058 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:35.822777 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-859ca77e-ad25-420d-9301-7e57d7400058 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:35.955265 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-859ca77e-ad25-420d-9301-7e57d7400058 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:35.965345 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-859ca77e-ad25-420d-9301-7e57d7400058 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1574 microversion: 2.1 time: 0.156600 Aug 08 02:32:35.966170 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 446/902] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:35 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1574 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:32:36.127819 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-1fa21f64-7b22-4cef-8937-fe8543e3599d tempest-VolumesGetTestJSON-107376284 tempest-VolumesGetTestJSON-107376284-project] Calling method '.version_select at 0x7efc6f7fc940>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:36.157659 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.wsgi [None req-1fa21f64-7b22-4cef-8937-fe8543e3599d tempest-VolumesGetTestJSON-107376284 tempest-VolumesGetTestJSON-107376284-project] HTTP exception thrown: Volume 405121b9-a73e-4dbd-8b5d-e5b850fc7dee could not be found. Aug 08 02:32:36.159594 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-1fa21f64-7b22-4cef-8937-fe8543e3599d tempest-VolumesGetTestJSON-107376284 tempest-VolumesGetTestJSON-107376284-project] Request method failure captured: Aug 08 02:32:36.159594 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: request: GET /compute/v2.1/os-volumes/405121b9-a73e-4dbd-8b5d-e5b850fc7dee HTTP/1.1 Aug 08 02:32:36.159594 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept: application/json Aug 08 02:32:36.159594 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept-Encoding: identity Aug 08 02:32:36.159594 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Connection: close Aug 08 02:32:36.159594 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Length: 0 Aug 08 02:32:36.159594 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Type: application/json Aug 08 02:32:36.159594 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Host: 10.222.0.27 Aug 08 02:32:36.159594 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Openstack-System-Scope: None Aug 08 02:32:36.159594 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: User-Agent: python-urllib3/1.26.11 Aug 08 02:32:36.159594 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Auth-Token: gAAAAABi8HXAKSfAqLNYtkJMihsrpE1jlrq5LgboXUb53_cviCpjAkvReg1AXjYUhgoXbbQIXx8QKPonWOwhZpJI3Cnc9Yo3cZXDkvKSKnYvrxYTp-JGsVPYyZP4oZwGRA1y-XgMhwPhThwWm9RC3YyyYyooDOj8k0UQsAjVMARwK3lxaT5IfN8 Aug 08 02:32:36.159594 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Id: None Aug 08 02:32:36.159594 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Name: None Aug 08 02:32:36.159594 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Identity-Status: Confirmed Aug 08 02:32:36.159594 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Is-Admin-Project: True Aug 08 02:32:36.159594 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Id: default Aug 08 02:32:36.159594 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Name: Default Aug 08 02:32:36.159594 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Id: dbe03b54834641bfbed01df88f536154 Aug 08 02:32:36.159594 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Name: tempest-VolumesGetTestJSON-107376284 Aug 08 02:32:36.159594 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Role: reader,member Aug 08 02:32:36.159594 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Roles: reader,member Aug 08 02:32:36.161151 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/dbe03b54834641bfbed01df88f536154"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/dbe03b54834641bfbed01df88f536154"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_dbe03b54834641bfbed01df88f536154", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/dbe03b54834641bfbed01df88f536154"}]}] Aug 08 02:32:36.161151 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant: tempest-VolumesGetTestJSON-107376284 Aug 08 02:32:36.161151 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Id: dbe03b54834641bfbed01df88f536154 Aug 08 02:32:36.161151 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Name: tempest-VolumesGetTestJSON-107376284 Aug 08 02:32:36.161151 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User: tempest-VolumesGetTestJSON-107376284-project Aug 08 02:32:36.161151 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Id: default Aug 08 02:32:36.161151 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Name: Default Aug 08 02:32:36.161151 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Id: 9a0912f1d353457c9bee96549f1af5a3 Aug 08 02:32:36.161151 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Name: tempest-VolumesGetTestJSON-107376284-project Aug 08 02:32:36.161151 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: method: Aug 08 02:32:36.161151 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: exception: Volume 405121b9-a73e-4dbd-8b5d-e5b850fc7dee could not be found. Aug 08 02:32:36.161151 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: action_args: {'id': '405121b9-a73e-4dbd-8b5d-e5b850fc7dee'} Aug 08 02:32:36.161151 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:32:36.162004 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:36.162004 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 08 02:32:36.162004 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 08 02:32:36.162004 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Aug 08 02:32:36.162004 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi item = cinderclient( Aug 08 02:32:36.162004 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Aug 08 02:32:36.162004 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Aug 08 02:32:36.162004 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 08 02:32:36.162004 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 08 02:32:36.162004 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 08 02:32:36.162004 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 08 02:32:36.162004 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 08 02:32:36.162004 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 08 02:32:36.162004 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 08 02:32:36.162004 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 08 02:32:36.162004 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 405121b9-a73e-4dbd-8b5d-e5b850fc7dee could not be found. (HTTP 404) (Request-ID: req-3eb63bb3-808d-41c3-9f42-0fe8e05e8c7f) Aug 08 02:32:36.162004 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:36.162004 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:32:36.162004 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:36.163166 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:36.163166 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Aug 08 02:32:36.163166 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Aug 08 02:32:36.163166 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 08 02:32:36.163166 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 08 02:32:36.163166 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Aug 08 02:32:36.163166 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Aug 08 02:32:36.163166 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 08 02:32:36.163166 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 08 02:32:36.163166 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 08 02:32:36.163166 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 08 02:32:36.163166 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Aug 08 02:32:36.163166 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi item = cinderclient( Aug 08 02:32:36.163166 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Aug 08 02:32:36.163166 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Aug 08 02:32:36.163166 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 08 02:32:36.163166 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 08 02:32:36.163166 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 08 02:32:36.163166 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 08 02:32:36.164296 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 08 02:32:36.164296 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 08 02:32:36.164296 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 08 02:32:36.164296 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 08 02:32:36.164296 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 405121b9-a73e-4dbd-8b5d-e5b850fc7dee could not be found. Aug 08 02:32:36.164296 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:36.164296 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:32:36.164296 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:36.164296 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:36.164296 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:32:36.164296 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:32:36.164296 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:32:36.164296 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:32:36.164296 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 08 02:32:36.164296 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 08 02:32:36.164296 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:32:36.164296 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:32:36.164296 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Aug 08 02:32:36.164296 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:32:36.165456 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 405121b9-a73e-4dbd-8b5d-e5b850fc7dee could not be found. Aug 08 02:32:36.165456 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:36.165456 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:32:36.165456 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:36.165456 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:36.165456 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:32:36.165456 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:32:36.165456 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:32:36.165456 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:32:36.165456 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 405121b9-a73e-4dbd-8b5d-e5b850fc7dee could not be found. Aug 08 02:32:36.165456 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:36.165456 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-1fa21f64-7b22-4cef-8937-fe8543e3599d tempest-VolumesGetTestJSON-107376284 tempest-VolumesGetTestJSON-107376284-project] Returning 404 to user: Volume 405121b9-a73e-4dbd-8b5d-e5b850fc7dee could not be found. {{(pid=118352) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:32:36.165456 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-1fa21f64-7b22-4cef-8937-fe8543e3599d tempest-VolumesGetTestJSON-107376284 tempest-VolumesGetTestJSON-107376284-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/405121b9-a73e-4dbd-8b5d-e5b850fc7dee" status: 404 len: 109 microversion: 2.1 time: 0.037673 Aug 08 02:32:36.165456 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 457/903] 10.222.0.27 () {58 vars in 1245 bytes} [Mon Aug 8 02:32:36 2022] GET /compute/v2.1/os-volumes/405121b9-a73e-4dbd-8b5d-e5b850fc7dee => generated 109 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 08 02:32:36.988257 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-64986938-69e7-4ce3-a3b8-db3b67826f82 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:36.989429 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-64986938-69e7-4ce3-a3b8-db3b67826f82 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:36.997081 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-64986938-69e7-4ce3-a3b8-db3b67826f82 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:36.997852 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-64986938-69e7-4ce3-a3b8-db3b67826f82 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:36.998287 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-64986938-69e7-4ce3-a3b8-db3b67826f82 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:36.999202 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-64986938-69e7-4ce3-a3b8-db3b67826f82 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:36.999850 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-64986938-69e7-4ce3-a3b8-db3b67826f82 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:37.000430 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-64986938-69e7-4ce3-a3b8-db3b67826f82 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:37.148228 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-64986938-69e7-4ce3-a3b8-db3b67826f82 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:37.156770 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-64986938-69e7-4ce3-a3b8-db3b67826f82 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1574 microversion: 2.1 time: 0.174177 Aug 08 02:32:37.157795 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 447/904] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:36 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1574 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:32:38.180006 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-76a014fc-113c-44ea-a974-da5da105319b tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:38.180767 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-76a014fc-113c-44ea-a974-da5da105319b tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:38.189724 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-76a014fc-113c-44ea-a974-da5da105319b tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:38.190301 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-76a014fc-113c-44ea-a974-da5da105319b tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:38.190742 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-76a014fc-113c-44ea-a974-da5da105319b tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:38.191659 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-76a014fc-113c-44ea-a974-da5da105319b tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:38.192130 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-76a014fc-113c-44ea-a974-da5da105319b tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:38.192561 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-76a014fc-113c-44ea-a974-da5da105319b tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:38.373056 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-76a014fc-113c-44ea-a974-da5da105319b tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:38.387668 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-76a014fc-113c-44ea-a974-da5da105319b tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1574 microversion: 2.1 time: 0.212128 Aug 08 02:32:38.389369 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 458/905] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:38 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1574 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:32:38.838267 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [req-1d614c3f-bd3b-4d8d-90da-2bf21d0e51a1 req-d9607ec5-dbc5-4a8c-b7d2-ff36028b2ae7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "077e7935-3361-4ff5-9367-ba162efac09c", "name": "network-vif-plugged", "status": "completed", "tag": "3051cabc-48a3-4055-ae74-0ae74aa73005"}]} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:32:38.848312 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-1d614c3f-bd3b-4d8d-90da-2bf21d0e51a1 req-d9607ec5-dbc5-4a8c-b7d2-ff36028b2ae7 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:38.848860 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-1d614c3f-bd3b-4d8d-90da-2bf21d0e51a1 req-d9607ec5-dbc5-4a8c-b7d2-ff36028b2ae7 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:38.849377 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-1d614c3f-bd3b-4d8d-90da-2bf21d0e51a1 req-d9607ec5-dbc5-4a8c-b7d2-ff36028b2ae7 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:38.853397 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:38.861357 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.compute.server_external_events [req-1d614c3f-bd3b-4d8d-90da-2bf21d0e51a1 req-d9607ec5-dbc5-4a8c-b7d2-ff36028b2ae7 service nova] Creating event network-vif-plugged:3051cabc-48a3-4055-ae74-0ae74aa73005 for instance 077e7935-3361-4ff5-9367-ba162efac09c on hv-cinder-74014 Aug 08 02:32:38.866197 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [req-1d614c3f-bd3b-4d8d-90da-2bf21d0e51a1 req-d9607ec5-dbc5-4a8c-b7d2-ff36028b2ae7 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031622 Aug 08 02:32:38.866769 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 448/906] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:32:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:32:39.411577 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-56748fbd-5c04-4f13-a49a-5e36af1a3e28 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:39.413509 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-56748fbd-5c04-4f13-a49a-5e36af1a3e28 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:39.422566 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-56748fbd-5c04-4f13-a49a-5e36af1a3e28 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:39.422566 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-56748fbd-5c04-4f13-a49a-5e36af1a3e28 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:39.422894 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-56748fbd-5c04-4f13-a49a-5e36af1a3e28 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:39.423771 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-56748fbd-5c04-4f13-a49a-5e36af1a3e28 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:39.424392 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-56748fbd-5c04-4f13-a49a-5e36af1a3e28 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:39.425078 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-56748fbd-5c04-4f13-a49a-5e36af1a3e28 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:39.588078 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-56748fbd-5c04-4f13-a49a-5e36af1a3e28 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:39.602756 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-56748fbd-5c04-4f13-a49a-5e36af1a3e28 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1574 microversion: 2.1 time: 0.195062 Aug 08 02:32:39.604127 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 459/907] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:39 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1574 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:32:40.627030 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-52270548-727c-4bac-85b7-a002a274e815 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:40.628625 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-52270548-727c-4bac-85b7-a002a274e815 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:40.636104 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-52270548-727c-4bac-85b7-a002a274e815 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:40.636690 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-52270548-727c-4bac-85b7-a002a274e815 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:40.637168 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-52270548-727c-4bac-85b7-a002a274e815 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:40.638003 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-52270548-727c-4bac-85b7-a002a274e815 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:40.638334 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-52270548-727c-4bac-85b7-a002a274e815 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:40.638758 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-52270548-727c-4bac-85b7-a002a274e815 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:40.778313 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-52270548-727c-4bac-85b7-a002a274e815 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:40.787478 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-52270548-727c-4bac-85b7-a002a274e815 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1574 microversion: 2.1 time: 0.163316 Aug 08 02:32:40.787478 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 449/908] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:40 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1574 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:32:41.807750 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-e8b10070-e19b-40d0-ab49-8c3a3c84741f tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:41.809156 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-e8b10070-e19b-40d0-ab49-8c3a3c84741f tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:41.815701 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e8b10070-e19b-40d0-ab49-8c3a3c84741f tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:41.816197 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e8b10070-e19b-40d0-ab49-8c3a3c84741f tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:41.816642 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e8b10070-e19b-40d0-ab49-8c3a3c84741f tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:41.817376 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e8b10070-e19b-40d0-ab49-8c3a3c84741f tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:41.817837 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e8b10070-e19b-40d0-ab49-8c3a3c84741f tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:41.818268 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e8b10070-e19b-40d0-ab49-8c3a3c84741f tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:41.966377 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-e8b10070-e19b-40d0-ab49-8c3a3c84741f tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:41.973998 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-e8b10070-e19b-40d0-ab49-8c3a3c84741f tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1561 microversion: 2.1 time: 0.170285 Aug 08 02:32:41.974878 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 460/909] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:41 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1561 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:32:45.015291 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-35d94664-445a-4d8a-a194-2c77affc06a7 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1055525620", "imageRef": "30833f47-5987-41ca-b5ee-ad5ecafbb265", "flavorRef": "452", "networks": [{"uuid": "d9455505-5caf-4786-81f8-4735ac0e318e"}]}} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:32:45.073700 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.network.neutron [None req-35d94664-445a-4d8a-a194-2c77affc06a7 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] validate_networks() for [('d9455505-5caf-4786-81f8-4735ac0e318e', None, None, None, None, None)] {{(pid=118351) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 08 02:32:45.573029 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.virt.hardware [None req-35d94664-445a-4d8a-a194-2c77affc06a7 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Flavor limits 0:0:0 {{(pid=118351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 08 02:32:45.574058 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.virt.hardware [None req-35d94664-445a-4d8a-a194-2c77affc06a7 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Image limits 0:0:0 {{(pid=118351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 08 02:32:45.574571 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.virt.hardware [None req-35d94664-445a-4d8a-a194-2c77affc06a7 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Flavor pref 0:0:0 {{(pid=118351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 08 02:32:45.575020 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.virt.hardware [None req-35d94664-445a-4d8a-a194-2c77affc06a7 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Image pref 0:0:0 {{(pid=118351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 08 02:32:45.575433 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.virt.hardware [None req-35d94664-445a-4d8a-a194-2c77affc06a7 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 08 02:32:45.584184 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-35d94664-445a-4d8a-a194-2c77affc06a7 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:45.584677 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-35d94664-445a-4d8a-a194-2c77affc06a7 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:45.585131 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-35d94664-445a-4d8a-a194-2c77affc06a7 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:45.585944 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-35d94664-445a-4d8a-a194-2c77affc06a7 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:45.586401 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-35d94664-445a-4d8a-a194-2c77affc06a7 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:45.586825 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-35d94664-445a-4d8a-a194-2c77affc06a7 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:45.604173 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.quota [None req-35d94664-445a-4d8a-a194-2c77affc06a7 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Getting quotas for project 414b6a313558425cbd3ed5c6da5405a9. Resources: {'ram', 'cores', 'instances'} {{(pid=118351) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 08 02:32:45.608486 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.quota [None req-35d94664-445a-4d8a-a194-2c77affc06a7 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Getting quotas for user ffcf7b8ddd454d9fa3b32f86906cedd5 and project 414b6a313558425cbd3ed5c6da5405a9. Resources: {'ram', 'cores', 'instances'} {{(pid=118351) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 08 02:32:45.619436 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-35d94664-445a-4d8a-a194-2c77affc06a7 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Going to run 1 instances... {{(pid=118351) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Aug 08 02:32:45.621752 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-35d94664-445a-4d8a-a194-2c77affc06a7 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] [instance: 209cb22c-fd1c-40f7-b091-43ef7195eabd] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='30833f47-5987-41ca-b5ee-ad5ecafbb265',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118351) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Aug 08 02:32:45.767205 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-35d94664-445a-4d8a-a194-2c77affc06a7 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.756435 Aug 08 02:32:45.767856 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 450/910] 10.222.0.27 () {60 vars in 1147 bytes} [Mon Aug 8 02:32:45 2022] POST /compute/v2.1/servers => generated 380 bytes in 758 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 08 02:32:45.780204 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-a6fdb3ce-cc32-4063-bbd0-bb5d01f46163 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:45.780733 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-a6fdb3ce-cc32-4063-bbd0-bb5d01f46163 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] [instance: 209cb22c-fd1c-40f7-b091-43ef7195eabd] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:45.838325 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-a6fdb3ce-cc32-4063-bbd0-bb5d01f46163 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ffcf7b8ddd454d9fa3b32f86906cedd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '414b6a313558425cbd3ed5c6da5405a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:45.847967 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-a6fdb3ce-cc32-4063-bbd0-bb5d01f46163 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] 10.222.0.27 "GET /compute/v2.1/servers/209cb22c-fd1c-40f7-b091-43ef7195eabd" status: 200 len: 1214 microversion: 2.1 time: 0.073181 Aug 08 02:32:45.848576 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 461/911] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:45 2022] GET /compute/v2.1/servers/209cb22c-fd1c-40f7-b091-43ef7195eabd => generated 1214 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:32:46.869118 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-75dc9f2f-4567-4afa-95cd-2e5d2e440e50 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:46.869591 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-75dc9f2f-4567-4afa-95cd-2e5d2e440e50 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] [instance: 209cb22c-fd1c-40f7-b091-43ef7195eabd] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:46.929798 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-75dc9f2f-4567-4afa-95cd-2e5d2e440e50 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ffcf7b8ddd454d9fa3b32f86906cedd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '414b6a313558425cbd3ed5c6da5405a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:46.937032 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-75dc9f2f-4567-4afa-95cd-2e5d2e440e50 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] 10.222.0.27 "GET /compute/v2.1/servers/209cb22c-fd1c-40f7-b091-43ef7195eabd" status: 200 len: 1214 microversion: 2.1 time: 0.072036 Aug 08 02:32:46.937605 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 451/912] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:46 2022] GET /compute/v2.1/servers/209cb22c-fd1c-40f7-b091-43ef7195eabd => generated 1214 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:32:47.958108 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-e5d59e09-dae6-45d9-84ea-d85c2be77142 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:47.958718 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-e5d59e09-dae6-45d9-84ea-d85c2be77142 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] [instance: 209cb22c-fd1c-40f7-b091-43ef7195eabd] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:47.961216 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_api:SELECT=54 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:47.967091 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e5d59e09-dae6-45d9-84ea-d85c2be77142 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:47.967581 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e5d59e09-dae6-45d9-84ea-d85c2be77142 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:47.968026 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e5d59e09-dae6-45d9-84ea-d85c2be77142 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:47.968741 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e5d59e09-dae6-45d9-84ea-d85c2be77142 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:47.969201 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e5d59e09-dae6-45d9-84ea-d85c2be77142 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:47.969630 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e5d59e09-dae6-45d9-84ea-d85c2be77142 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:48.036487 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-e5d59e09-dae6-45d9-84ea-d85c2be77142 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ffcf7b8ddd454d9fa3b32f86906cedd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '414b6a313558425cbd3ed5c6da5405a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:48.044856 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-e5d59e09-dae6-45d9-84ea-d85c2be77142 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] 10.222.0.27 "GET /compute/v2.1/servers/209cb22c-fd1c-40f7-b091-43ef7195eabd" status: 200 len: 1266 microversion: 2.1 time: 0.090632 Aug 08 02:32:48.045438 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 462/913] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:47 2022] GET /compute/v2.1/servers/209cb22c-fd1c-40f7-b091-43ef7195eabd => generated 1266 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:32:48.817909 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-f5c4f272-1a56-4d34-822e-d5f9e29ea4c6 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:48.820095 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-f5c4f272-1a56-4d34-822e-d5f9e29ea4c6 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:48.827303 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f5c4f272-1a56-4d34-822e-d5f9e29ea4c6 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:48.827788 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f5c4f272-1a56-4d34-822e-d5f9e29ea4c6 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:48.828217 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f5c4f272-1a56-4d34-822e-d5f9e29ea4c6 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:48.828976 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f5c4f272-1a56-4d34-822e-d5f9e29ea4c6 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:48.829425 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f5c4f272-1a56-4d34-822e-d5f9e29ea4c6 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:48.829846 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f5c4f272-1a56-4d34-822e-d5f9e29ea4c6 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:48.860768 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-f5c4f272-1a56-4d34-822e-d5f9e29ea4c6 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.046993 Aug 08 02:32:48.861337 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 452/914] 10.222.0.27 () {60 vars in 1343 bytes} [Mon Aug 8 02:32:48 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d/os-volume_attachments => generated 25 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:32:48.926768 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-42049527-9a1c-42d9-b8fb-c6bf51675415 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:48.927542 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-42049527-9a1c-42d9-b8fb-c6bf51675415 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:48.935758 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-42049527-9a1c-42d9-b8fb-c6bf51675415 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:48.940480 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-42049527-9a1c-42d9-b8fb-c6bf51675415 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:48.941965 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-42049527-9a1c-42d9-b8fb-c6bf51675415 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:48.942751 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-42049527-9a1c-42d9-b8fb-c6bf51675415 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:48.943242 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-42049527-9a1c-42d9-b8fb-c6bf51675415 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:48.943681 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-42049527-9a1c-42d9-b8fb-c6bf51675415 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:48.983380 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-42049527-9a1c-42d9-b8fb-c6bf51675415 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Going to try to terminate instance {{(pid=118352) delete /opt/stack/nova/nova/compute/api.py:2666}} Aug 08 02:32:49.067445 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-11fb1b7f-6cf4-46cf-9b0a-c7504cfd3550 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:49.068068 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-11fb1b7f-6cf4-46cf-9b0a-c7504cfd3550 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] [instance: 209cb22c-fd1c-40f7-b091-43ef7195eabd] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:49.075083 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-11fb1b7f-6cf4-46cf-9b0a-c7504cfd3550 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:49.075574 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-11fb1b7f-6cf4-46cf-9b0a-c7504cfd3550 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:49.076013 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-11fb1b7f-6cf4-46cf-9b0a-c7504cfd3550 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:49.076736 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-11fb1b7f-6cf4-46cf-9b0a-c7504cfd3550 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:49.077194 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-11fb1b7f-6cf4-46cf-9b0a-c7504cfd3550 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:49.077670 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-11fb1b7f-6cf4-46cf-9b0a-c7504cfd3550 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:49.141159 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-11fb1b7f-6cf4-46cf-9b0a-c7504cfd3550 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ffcf7b8ddd454d9fa3b32f86906cedd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '414b6a313558425cbd3ed5c6da5405a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:49.149884 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-11fb1b7f-6cf4-46cf-9b0a-c7504cfd3550 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] 10.222.0.27 "GET /compute/v2.1/servers/209cb22c-fd1c-40f7-b091-43ef7195eabd" status: 200 len: 1284 microversion: 2.1 time: 0.087144 Aug 08 02:32:49.150448 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 453/915] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:49 2022] GET /compute/v2.1/servers/209cb22c-fd1c-40f7-b091-43ef7195eabd => generated 1284 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:32:49.180686 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell1:SELECT=99,nova_cell1:UPDATE=1 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:49.481350 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-42049527-9a1c-42d9-b8fb-c6bf51675415 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "DELETE /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 204 len: 0 microversion: 2.20 time: 0.558505 Aug 08 02:32:49.484380 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 463/916] 10.222.0.27 () {60 vars in 1280 bytes} [Mon Aug 8 02:32:48 2022] DELETE /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 0 bytes in 562 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Aug 08 02:32:49.492804 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-6d96d667-7ae9-401b-903a-4b592fa34050 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:49.493415 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-6d96d667-7ae9-401b-903a-4b592fa34050 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:49.500298 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6d96d667-7ae9-401b-903a-4b592fa34050 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:49.500829 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6d96d667-7ae9-401b-903a-4b592fa34050 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:49.501257 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6d96d667-7ae9-401b-903a-4b592fa34050 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:49.501951 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6d96d667-7ae9-401b-903a-4b592fa34050 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:49.502404 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6d96d667-7ae9-401b-903a-4b592fa34050 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:49.502827 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6d96d667-7ae9-401b-903a-4b592fa34050 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:49.634766 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-6d96d667-7ae9-401b-903a-4b592fa34050 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:49.642912 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-6d96d667-7ae9-401b-903a-4b592fa34050 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:49.644603 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-6d96d667-7ae9-401b-903a-4b592fa34050 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:49.645614 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-6d96d667-7ae9-401b-903a-4b592fa34050 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1723 microversion: 2.20 time: 0.156792 Aug 08 02:32:49.646169 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 454/917] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:32:49 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1723 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:32:50.171745 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-00fcfe8f-db11-4177-bcaf-caffa08b2973 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:50.172447 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-00fcfe8f-db11-4177-bcaf-caffa08b2973 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] [instance: 209cb22c-fd1c-40f7-b091-43ef7195eabd] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:50.178825 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-00fcfe8f-db11-4177-bcaf-caffa08b2973 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:50.179237 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-00fcfe8f-db11-4177-bcaf-caffa08b2973 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:50.179676 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-00fcfe8f-db11-4177-bcaf-caffa08b2973 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:50.180363 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-00fcfe8f-db11-4177-bcaf-caffa08b2973 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:50.180856 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-00fcfe8f-db11-4177-bcaf-caffa08b2973 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:50.181287 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-00fcfe8f-db11-4177-bcaf-caffa08b2973 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:50.315567 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-00fcfe8f-db11-4177-bcaf-caffa08b2973 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ffcf7b8ddd454d9fa3b32f86906cedd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '414b6a313558425cbd3ed5c6da5405a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:50.323794 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-00fcfe8f-db11-4177-bcaf-caffa08b2973 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] 10.222.0.27 "GET /compute/v2.1/servers/209cb22c-fd1c-40f7-b091-43ef7195eabd" status: 200 len: 1314 microversion: 2.1 time: 0.155975 Aug 08 02:32:50.324969 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 464/918] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:50 2022] GET /compute/v2.1/servers/209cb22c-fd1c-40f7-b091-43ef7195eabd => generated 1314 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:32:50.670292 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-7e3f9b2b-8133-4e7f-b9f4-2aeb7e6fe8d8 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:50.670922 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-7e3f9b2b-8133-4e7f-b9f4-2aeb7e6fe8d8 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:50.681053 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7e3f9b2b-8133-4e7f-b9f4-2aeb7e6fe8d8 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:50.681542 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7e3f9b2b-8133-4e7f-b9f4-2aeb7e6fe8d8 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:50.681975 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7e3f9b2b-8133-4e7f-b9f4-2aeb7e6fe8d8 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:50.683568 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7e3f9b2b-8133-4e7f-b9f4-2aeb7e6fe8d8 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:50.684178 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7e3f9b2b-8133-4e7f-b9f4-2aeb7e6fe8d8 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:50.684607 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7e3f9b2b-8133-4e7f-b9f4-2aeb7e6fe8d8 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:50.818419 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-7e3f9b2b-8133-4e7f-b9f4-2aeb7e6fe8d8 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:50.829048 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-7e3f9b2b-8133-4e7f-b9f4-2aeb7e6fe8d8 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:50.830685 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-7e3f9b2b-8133-4e7f-b9f4-2aeb7e6fe8d8 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:50.832240 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-7e3f9b2b-8133-4e7f-b9f4-2aeb7e6fe8d8 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1723 microversion: 2.20 time: 0.166225 Aug 08 02:32:50.833116 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 455/919] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:32:50 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1723 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:32:51.345152 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-bdf46ea7-2e0c-414e-9e22-bd9810efdc07 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:51.346842 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-bdf46ea7-2e0c-414e-9e22-bd9810efdc07 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] [instance: 209cb22c-fd1c-40f7-b091-43ef7195eabd] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:51.352636 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-bdf46ea7-2e0c-414e-9e22-bd9810efdc07 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:51.353217 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-bdf46ea7-2e0c-414e-9e22-bd9810efdc07 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:51.353681 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-bdf46ea7-2e0c-414e-9e22-bd9810efdc07 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:51.354381 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-bdf46ea7-2e0c-414e-9e22-bd9810efdc07 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:51.355495 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-bdf46ea7-2e0c-414e-9e22-bd9810efdc07 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:51.356082 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-bdf46ea7-2e0c-414e-9e22-bd9810efdc07 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:51.497431 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-bdf46ea7-2e0c-414e-9e22-bd9810efdc07 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ffcf7b8ddd454d9fa3b32f86906cedd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '414b6a313558425cbd3ed5c6da5405a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:51.505634 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-bdf46ea7-2e0c-414e-9e22-bd9810efdc07 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] 10.222.0.27 "GET /compute/v2.1/servers/209cb22c-fd1c-40f7-b091-43ef7195eabd" status: 200 len: 1314 microversion: 2.1 time: 0.164339 Aug 08 02:32:51.506337 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 465/920] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:51 2022] GET /compute/v2.1/servers/209cb22c-fd1c-40f7-b091-43ef7195eabd => generated 1314 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:32:51.858013 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-5b67440c-a47b-4a37-a6f7-b6e8ea1f7201 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:51.858582 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-5b67440c-a47b-4a37-a6f7-b6e8ea1f7201 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:51.865276 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5b67440c-a47b-4a37-a6f7-b6e8ea1f7201 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:51.865909 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5b67440c-a47b-4a37-a6f7-b6e8ea1f7201 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:51.868200 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5b67440c-a47b-4a37-a6f7-b6e8ea1f7201 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:51.868200 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5b67440c-a47b-4a37-a6f7-b6e8ea1f7201 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:51.868200 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5b67440c-a47b-4a37-a6f7-b6e8ea1f7201 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:51.868200 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5b67440c-a47b-4a37-a6f7-b6e8ea1f7201 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:52.007115 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-5b67440c-a47b-4a37-a6f7-b6e8ea1f7201 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:52.017543 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-5b67440c-a47b-4a37-a6f7-b6e8ea1f7201 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:52.019125 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-5b67440c-a47b-4a37-a6f7-b6e8ea1f7201 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:52.020523 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-5b67440c-a47b-4a37-a6f7-b6e8ea1f7201 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1723 microversion: 2.20 time: 0.168559 Aug 08 02:32:52.021091 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 456/921] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:32:51 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1723 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:32:52.022125 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [req-f4109e5a-1483-48fd-9596-e10da1412ad0 req-e29c95a7-e9b1-4035-b7f0-edc413f1f64f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "209cb22c-fd1c-40f7-b091-43ef7195eabd", "tag": "706064e3-1136-4ec7-9c05-e6e912a96ad2"}]} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:32:52.042657 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-f4109e5a-1483-48fd-9596-e10da1412ad0 req-e29c95a7-e9b1-4035-b7f0-edc413f1f64f service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:52.043248 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-f4109e5a-1483-48fd-9596-e10da1412ad0 req-e29c95a7-e9b1-4035-b7f0-edc413f1f64f service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:52.043695 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-f4109e5a-1483-48fd-9596-e10da1412ad0 req-e29c95a7-e9b1-4035-b7f0-edc413f1f64f service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:52.056595 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.compute.server_external_events [req-f4109e5a-1483-48fd-9596-e10da1412ad0 req-e29c95a7-e9b1-4035-b7f0-edc413f1f64f service nova] Creating event network-changed:706064e3-1136-4ec7-9c05-e6e912a96ad2 for instance 209cb22c-fd1c-40f7-b091-43ef7195eabd on hv-cinder-74014 Aug 08 02:32:52.065971 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [req-f4109e5a-1483-48fd-9596-e10da1412ad0 req-e29c95a7-e9b1-4035-b7f0-edc413f1f64f service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.043529 Aug 08 02:32:52.065971 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 466/922] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:32:52 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 08 02:32:52.534611 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-b4396841-29b2-4208-aaf2-5cf1e2fce075 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:52.535227 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-b4396841-29b2-4208-aaf2-5cf1e2fce075 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] [instance: 209cb22c-fd1c-40f7-b091-43ef7195eabd] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:52.543798 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b4396841-29b2-4208-aaf2-5cf1e2fce075 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:52.544291 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b4396841-29b2-4208-aaf2-5cf1e2fce075 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:52.544745 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b4396841-29b2-4208-aaf2-5cf1e2fce075 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:52.545443 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b4396841-29b2-4208-aaf2-5cf1e2fce075 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:52.546021 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b4396841-29b2-4208-aaf2-5cf1e2fce075 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:52.546625 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b4396841-29b2-4208-aaf2-5cf1e2fce075 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:52.718839 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-b4396841-29b2-4208-aaf2-5cf1e2fce075 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ffcf7b8ddd454d9fa3b32f86906cedd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '414b6a313558425cbd3ed5c6da5405a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:52.728298 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-b4396841-29b2-4208-aaf2-5cf1e2fce075 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] 10.222.0.27 "GET /compute/v2.1/servers/209cb22c-fd1c-40f7-b091-43ef7195eabd" status: 200 len: 1314 microversion: 2.1 time: 0.199281 Aug 08 02:32:52.729115 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 457/923] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:52 2022] GET /compute/v2.1/servers/209cb22c-fd1c-40f7-b091-43ef7195eabd => generated 1314 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:32:53.051406 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-4165f241-46e1-4a4d-aaab-40678d6471fe tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:53.052321 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-4165f241-46e1-4a4d-aaab-40678d6471fe tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:53.059413 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4165f241-46e1-4a4d-aaab-40678d6471fe tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:53.059969 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4165f241-46e1-4a4d-aaab-40678d6471fe tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:53.060616 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4165f241-46e1-4a4d-aaab-40678d6471fe tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:53.061348 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4165f241-46e1-4a4d-aaab-40678d6471fe tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:53.061813 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4165f241-46e1-4a4d-aaab-40678d6471fe tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:53.062241 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4165f241-46e1-4a4d-aaab-40678d6471fe tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:53.094684 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4165f241-46e1-4a4d-aaab-40678d6471fe tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:53.095177 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4165f241-46e1-4a4d-aaab-40678d6471fe tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:53.095613 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4165f241-46e1-4a4d-aaab-40678d6471fe tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:53.096065 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.objects.instance [None req-4165f241-46e1-4a4d-aaab-40678d6471fe tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lazy-loading 'fault' on Instance uuid 946fd571-bd98-44f9-89f4-52f0116b642d {{(pid=118352) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 08 02:32:53.142515 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-4165f241-46e1-4a4d-aaab-40678d6471fe tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:53.150486 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-4165f241-46e1-4a4d-aaab-40678d6471fe tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:53.152145 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-4165f241-46e1-4a4d-aaab-40678d6471fe tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:53.153265 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-4165f241-46e1-4a4d-aaab-40678d6471fe tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1370 microversion: 2.20 time: 0.107629 Aug 08 02:32:53.153835 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 467/924] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:32:53 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1370 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:32:53.750039 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-97695691-49f5-4103-b148-5a249d825e16 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:53.750678 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-97695691-49f5-4103-b148-5a249d825e16 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] [instance: 209cb22c-fd1c-40f7-b091-43ef7195eabd] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:53.757290 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-97695691-49f5-4103-b148-5a249d825e16 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:53.757777 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-97695691-49f5-4103-b148-5a249d825e16 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:53.758213 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-97695691-49f5-4103-b148-5a249d825e16 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:53.758915 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-97695691-49f5-4103-b148-5a249d825e16 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:53.759369 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-97695691-49f5-4103-b148-5a249d825e16 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:53.759794 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-97695691-49f5-4103-b148-5a249d825e16 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:53.888918 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-97695691-49f5-4103-b148-5a249d825e16 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ffcf7b8ddd454d9fa3b32f86906cedd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '414b6a313558425cbd3ed5c6da5405a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:53.896439 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-97695691-49f5-4103-b148-5a249d825e16 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] 10.222.0.27 "GET /compute/v2.1/servers/209cb22c-fd1c-40f7-b091-43ef7195eabd" status: 200 len: 1314 microversion: 2.1 time: 0.150326 Aug 08 02:32:53.897040 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 458/925] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:53 2022] GET /compute/v2.1/servers/209cb22c-fd1c-40f7-b091-43ef7195eabd => generated 1314 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:32:54.077357 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [req-f2ceae4d-5f59-4c21-aa64-42025bae5220 req-e8d589a7-0116-4871-a839-2155cf99565c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "946fd571-bd98-44f9-89f4-52f0116b642d", "name": "network-vif-deleted", "tag": "0bf885ea-ca94-45b2-aafa-7461404e6cf2"}]} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:32:54.087855 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-f2ceae4d-5f59-4c21-aa64-42025bae5220 req-e8d589a7-0116-4871-a839-2155cf99565c service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:54.088726 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-f2ceae4d-5f59-4c21-aa64-42025bae5220 req-e8d589a7-0116-4871-a839-2155cf99565c service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:54.089172 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-f2ceae4d-5f59-4c21-aa64-42025bae5220 req-e8d589a7-0116-4871-a839-2155cf99565c service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:54.103800 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.compute.server_external_events [req-f2ceae4d-5f59-4c21-aa64-42025bae5220 req-e8d589a7-0116-4871-a839-2155cf99565c service nova] Creating event network-vif-deleted:0bf885ea-ca94-45b2-aafa-7461404e6cf2 for instance 946fd571-bd98-44f9-89f4-52f0116b642d on hv-cinder-74014 Aug 08 02:32:54.109740 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [req-f2ceae4d-5f59-4c21-aa64-42025bae5220 req-e8d589a7-0116-4871-a839-2155cf99565c service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036525 Aug 08 02:32:54.110346 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 468/926] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:32:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 08 02:32:54.175734 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-0a459caf-4e0a-4e06-a708-5a82bceb190f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:54.176347 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-0a459caf-4e0a-4e06-a708-5a82bceb190f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:54.183253 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0a459caf-4e0a-4e06-a708-5a82bceb190f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:54.183780 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0a459caf-4e0a-4e06-a708-5a82bceb190f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:54.184215 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0a459caf-4e0a-4e06-a708-5a82bceb190f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:54.185537 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0a459caf-4e0a-4e06-a708-5a82bceb190f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:54.186000 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0a459caf-4e0a-4e06-a708-5a82bceb190f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:54.186635 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0a459caf-4e0a-4e06-a708-5a82bceb190f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:54.221312 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0a459caf-4e0a-4e06-a708-5a82bceb190f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:54.221800 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0a459caf-4e0a-4e06-a708-5a82bceb190f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:54.222235 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0a459caf-4e0a-4e06-a708-5a82bceb190f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:54.222706 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.objects.instance [None req-0a459caf-4e0a-4e06-a708-5a82bceb190f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lazy-loading 'fault' on Instance uuid 946fd571-bd98-44f9-89f4-52f0116b642d {{(pid=118351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 08 02:32:54.264491 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-0a459caf-4e0a-4e06-a708-5a82bceb190f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:54.272444 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-0a459caf-4e0a-4e06-a708-5a82bceb190f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:54.273985 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-0a459caf-4e0a-4e06-a708-5a82bceb190f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd451fb2f26ce4f50b010fb2a387c4bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:54.275182 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-0a459caf-4e0a-4e06-a708-5a82bceb190f tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 200 len: 1370 microversion: 2.20 time: 0.103625 Aug 08 02:32:54.275728 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 459/927] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:32:54 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 1370 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 08 02:32:54.917779 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-b23d6647-12f1-43ef-a24f-38b0437d9946 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:54.918478 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-b23d6647-12f1-43ef-a24f-38b0437d9946 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] [instance: 209cb22c-fd1c-40f7-b091-43ef7195eabd] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:54.924639 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b23d6647-12f1-43ef-a24f-38b0437d9946 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:54.925150 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b23d6647-12f1-43ef-a24f-38b0437d9946 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:54.925590 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b23d6647-12f1-43ef-a24f-38b0437d9946 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:54.926279 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b23d6647-12f1-43ef-a24f-38b0437d9946 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:54.926742 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b23d6647-12f1-43ef-a24f-38b0437d9946 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:54.927166 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b23d6647-12f1-43ef-a24f-38b0437d9946 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:55.061674 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-b23d6647-12f1-43ef-a24f-38b0437d9946 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ffcf7b8ddd454d9fa3b32f86906cedd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '414b6a313558425cbd3ed5c6da5405a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:55.069095 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-b23d6647-12f1-43ef-a24f-38b0437d9946 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] 10.222.0.27 "GET /compute/v2.1/servers/209cb22c-fd1c-40f7-b091-43ef7195eabd" status: 200 len: 1314 microversion: 2.1 time: 0.155439 Aug 08 02:32:55.069650 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 469/928] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:54 2022] GET /compute/v2.1/servers/209cb22c-fd1c-40f7-b091-43ef7195eabd => generated 1314 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:32:55.299052 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-30362fa4-850f-42a6-9ce4-b23cccc79cb5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:55.299902 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-30362fa4-850f-42a6-9ce4-b23cccc79cb5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:55.302250 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_api:SELECT=65,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:55.308966 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-30362fa4-850f-42a6-9ce4-b23cccc79cb5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:55.309251 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-30362fa4-850f-42a6-9ce4-b23cccc79cb5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:55.309753 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-30362fa4-850f-42a6-9ce4-b23cccc79cb5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:55.310402 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-30362fa4-850f-42a6-9ce4-b23cccc79cb5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:55.310810 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-30362fa4-850f-42a6-9ce4-b23cccc79cb5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:55.311355 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-30362fa4-850f-42a6-9ce4-b23cccc79cb5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:55.326630 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.wsgi [None req-30362fa4-850f-42a6-9ce4-b23cccc79cb5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] HTTP exception thrown: Instance 946fd571-bd98-44f9-89f4-52f0116b642d could not be found. Aug 08 02:32:55.328347 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-30362fa4-850f-42a6-9ce4-b23cccc79cb5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Request method failure captured: Aug 08 02:32:55.328347 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: request: GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d HTTP/1.1 Aug 08 02:32:55.328347 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Accept: application/json Aug 08 02:32:55.328347 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Accept-Encoding: identity Aug 08 02:32:55.328347 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Connection: close Aug 08 02:32:55.328347 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Content-Length: 0 Aug 08 02:32:55.328347 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Content-Type: application/json Aug 08 02:32:55.328347 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Host: 10.222.0.27 Aug 08 02:32:55.328347 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Openstack-System-Scope: None Aug 08 02:32:55.328347 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: User-Agent: python-urllib3/1.26.11 Aug 08 02:32:55.328347 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Auth-Token: gAAAAABi8HQZM1u28QkO94W5_BI9_CK6trquSemX1B1L_Z-hB5Xr_ijBdmU9-G9fQkpWPZoqRyWzMs8DIWVoaDURV27YVJYuW-ZoqYszcoIWuLgl1q8SendZ4PZ3TqxVgh6HOFphSeT7fOMCYwtBnDGaMK-ygmFXZVrpyqVNef3pWtqKHJNgdaU Aug 08 02:32:55.328347 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Domain-Id: None Aug 08 02:32:55.328347 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Domain-Name: None Aug 08 02:32:55.328347 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Identity-Status: Confirmed Aug 08 02:32:55.328347 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Is-Admin-Project: True Aug 08 02:32:55.328347 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Openstack-Nova-Api-Version: 2.20 Aug 08 02:32:55.328347 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Domain-Id: default Aug 08 02:32:55.328347 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Domain-Name: Default Aug 08 02:32:55.328347 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Id: 7caa43bf43ad4e84b4b648e9ab33bd7f Aug 08 02:32:55.328347 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1887360550 Aug 08 02:32:55.328347 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Role: reader,member Aug 08 02:32:55.328347 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Roles: reader,member Aug 08 02:32:55.330183 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/7caa43bf43ad4e84b4b648e9ab33bd7f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/7caa43bf43ad4e84b4b648e9ab33bd7f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_7caa43bf43ad4e84b4b648e9ab33bd7f", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/7caa43bf43ad4e84b4b648e9ab33bd7f"}]}] Aug 08 02:32:55.330183 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1887360550 Aug 08 02:32:55.330183 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant-Id: 7caa43bf43ad4e84b4b648e9ab33bd7f Aug 08 02:32:55.330183 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1887360550 Aug 08 02:32:55.330183 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User: tempest-AttachVolumeShelveTestJSON-1887360550-project Aug 08 02:32:55.330183 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Domain-Id: default Aug 08 02:32:55.330183 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Domain-Name: Default Aug 08 02:32:55.330183 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Id: d451fb2f26ce4f50b010fb2a387c4bf3 Aug 08 02:32:55.330183 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1887360550-project Aug 08 02:32:55.330183 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: method: > Aug 08 02:32:55.330183 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: exception: Instance 946fd571-bd98-44f9-89f4-52f0116b642d could not be found. Aug 08 02:32:55.330183 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: action_args: {'id': '946fd571-bd98-44f9-89f4-52f0116b642d'} Aug 08 02:32:55.331096 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:32:55.331096 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:55.331096 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 08 02:32:55.331096 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 08 02:32:55.331096 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 08 02:32:55.331096 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 08 02:32:55.331096 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 08 02:32:55.331096 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 08 02:32:55.331096 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 08 02:32:55.331096 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 08 02:32:55.331096 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 946fd571-bd98-44f9-89f4-52f0116b642d could not be found. Aug 08 02:32:55.331096 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:32:55.331096 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:32:55.331096 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:32:55.331096 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:55.331096 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:32:55.331096 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:32:55.331096 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:32:55.331096 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:32:55.331096 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:32:55.332711 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:32:55.332711 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 08 02:32:55.332711 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 08 02:32:55.332711 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 08 02:32:55.332711 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 08 02:32:55.332711 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 08 02:32:55.332711 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:32:55.332711 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 946fd571-bd98-44f9-89f4-52f0116b642d could not be found. Aug 08 02:32:55.332711 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:32:55.332711 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:32:55.332711 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:32:55.332711 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:55.332711 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:32:55.332711 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:32:55.332711 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:32:55.332711 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:32:55.332711 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 946fd571-bd98-44f9-89f4-52f0116b642d could not be found. Aug 08 02:32:55.332711 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:32:55.334939 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-30362fa4-850f-42a6-9ce4-b23cccc79cb5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Returning 404 to user: Instance 946fd571-bd98-44f9-89f4-52f0116b642d could not be found. {{(pid=118351) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:32:55.334939 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-30362fa4-850f-42a6-9ce4-b23cccc79cb5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 404 len: 111 microversion: 2.20 time: 0.035312 Aug 08 02:32:55.334939 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 460/929] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:32:55 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Aug 08 02:32:55.339441 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-36cb8fc4-1082-4166-b7f9-658c95549734 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '.version_select at 0x7efc740813a0>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:55.442845 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-36cb8fc4-1082-4166-b7f9-658c95549734 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-902346280" status: 204 len: 0 microversion: 2.20 time: 0.107480 Aug 08 02:32:55.451190 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 470/930] 10.222.0.27 () {60 vars in 1259 bytes} [Mon Aug 8 02:32:55 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-902346280 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Aug 08 02:32:56.092097 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-09acea37-2e63-416e-8f7e-c2e4f72675c9 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:56.092742 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-09acea37-2e63-416e-8f7e-c2e4f72675c9 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] [instance: 209cb22c-fd1c-40f7-b091-43ef7195eabd] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:56.094558 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell0:SELECT=2 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:56.099760 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-09acea37-2e63-416e-8f7e-c2e4f72675c9 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:56.100245 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-09acea37-2e63-416e-8f7e-c2e4f72675c9 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:56.100695 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-09acea37-2e63-416e-8f7e-c2e4f72675c9 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:56.101395 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-09acea37-2e63-416e-8f7e-c2e4f72675c9 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:56.101848 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-09acea37-2e63-416e-8f7e-c2e4f72675c9 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:56.102276 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-09acea37-2e63-416e-8f7e-c2e4f72675c9 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:56.235393 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-09acea37-2e63-416e-8f7e-c2e4f72675c9 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ffcf7b8ddd454d9fa3b32f86906cedd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '414b6a313558425cbd3ed5c6da5405a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:56.242920 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-09acea37-2e63-416e-8f7e-c2e4f72675c9 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] 10.222.0.27 "GET /compute/v2.1/servers/209cb22c-fd1c-40f7-b091-43ef7195eabd" status: 200 len: 1314 microversion: 2.1 time: 0.155905 Aug 08 02:32:56.243540 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 461/931] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:56 2022] GET /compute/v2.1/servers/209cb22c-fd1c-40f7-b091-43ef7195eabd => generated 1314 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:32:57.084364 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [req-e52d44f1-5e56-4603-ae58-4ae868466c92 req-ba5de2c4-20ac-45ec-8606-653e1286cf4a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "209cb22c-fd1c-40f7-b091-43ef7195eabd", "name": "network-vif-plugged", "status": "completed", "tag": "706064e3-1136-4ec7-9c05-e6e912a96ad2"}]} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:32:57.089338 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell0:SELECT=2 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:57.096281 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-e52d44f1-5e56-4603-ae58-4ae868466c92 req-ba5de2c4-20ac-45ec-8606-653e1286cf4a service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:57.096798 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-e52d44f1-5e56-4603-ae58-4ae868466c92 req-ba5de2c4-20ac-45ec-8606-653e1286cf4a service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:57.097236 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-e52d44f1-5e56-4603-ae58-4ae868466c92 req-ba5de2c4-20ac-45ec-8606-653e1286cf4a service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:57.108147 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.compute.server_external_events [req-e52d44f1-5e56-4603-ae58-4ae868466c92 req-ba5de2c4-20ac-45ec-8606-653e1286cf4a service nova] Creating event network-vif-plugged:706064e3-1136-4ec7-9c05-e6e912a96ad2 for instance 209cb22c-fd1c-40f7-b091-43ef7195eabd on hv-cinder-74014 Aug 08 02:32:57.114243 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [req-e52d44f1-5e56-4603-ae58-4ae868466c92 req-ba5de2c4-20ac-45ec-8606-653e1286cf4a service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033299 Aug 08 02:32:57.116277 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 471/932] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:32:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:32:57.264918 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-729255b5-4776-4786-8ebb-f1386d61edb6 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:57.267387 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-729255b5-4776-4786-8ebb-f1386d61edb6 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] [instance: 209cb22c-fd1c-40f7-b091-43ef7195eabd] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:57.270679 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell0:SELECT=8 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:57.277131 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-729255b5-4776-4786-8ebb-f1386d61edb6 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:57.277747 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-729255b5-4776-4786-8ebb-f1386d61edb6 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:57.282660 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-729255b5-4776-4786-8ebb-f1386d61edb6 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:57.282660 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-729255b5-4776-4786-8ebb-f1386d61edb6 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:57.282660 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-729255b5-4776-4786-8ebb-f1386d61edb6 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:57.282660 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-729255b5-4776-4786-8ebb-f1386d61edb6 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:57.417961 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-729255b5-4776-4786-8ebb-f1386d61edb6 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ffcf7b8ddd454d9fa3b32f86906cedd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '414b6a313558425cbd3ed5c6da5405a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:57.424485 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-729255b5-4776-4786-8ebb-f1386d61edb6 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] 10.222.0.27 "GET /compute/v2.1/servers/209cb22c-fd1c-40f7-b091-43ef7195eabd" status: 200 len: 1314 microversion: 2.1 time: 0.164918 Aug 08 02:32:57.431984 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 462/933] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:57 2022] GET /compute/v2.1/servers/209cb22c-fd1c-40f7-b091-43ef7195eabd => generated 1314 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:32:57.778005 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-67823d39-97cb-4428-9f98-56507d075739 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:57.782442 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-67823d39-97cb-4428-9f98-56507d075739 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:57.785285 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-67823d39-97cb-4428-9f98-56507d075739 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:57.785775 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-67823d39-97cb-4428-9f98-56507d075739 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:57.786213 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-67823d39-97cb-4428-9f98-56507d075739 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:57.786903 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-67823d39-97cb-4428-9f98-56507d075739 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:57.787359 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-67823d39-97cb-4428-9f98-56507d075739 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:57.787788 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-67823d39-97cb-4428-9f98-56507d075739 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:57.801569 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.wsgi [None req-67823d39-97cb-4428-9f98-56507d075739 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] HTTP exception thrown: Instance 946fd571-bd98-44f9-89f4-52f0116b642d could not be found. Aug 08 02:32:57.803210 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-67823d39-97cb-4428-9f98-56507d075739 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Request method failure captured: Aug 08 02:32:57.803210 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: request: DELETE /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d HTTP/1.1 Aug 08 02:32:57.803210 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept: application/json Aug 08 02:32:57.803210 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept-Encoding: identity Aug 08 02:32:57.803210 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Connection: close Aug 08 02:32:57.803210 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Length: 0 Aug 08 02:32:57.803210 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Type: application/json Aug 08 02:32:57.803210 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Host: 10.222.0.27 Aug 08 02:32:57.803210 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Openstack-System-Scope: None Aug 08 02:32:57.803210 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: User-Agent: python-urllib3/1.26.11 Aug 08 02:32:57.803210 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Auth-Token: gAAAAABi8HQZM1u28QkO94W5_BI9_CK6trquSemX1B1L_Z-hB5Xr_ijBdmU9-G9fQkpWPZoqRyWzMs8DIWVoaDURV27YVJYuW-ZoqYszcoIWuLgl1q8SendZ4PZ3TqxVgh6HOFphSeT7fOMCYwtBnDGaMK-ygmFXZVrpyqVNef3pWtqKHJNgdaU Aug 08 02:32:57.803210 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Id: None Aug 08 02:32:57.803210 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Name: None Aug 08 02:32:57.803210 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Identity-Status: Confirmed Aug 08 02:32:57.803210 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Is-Admin-Project: True Aug 08 02:32:57.803210 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Openstack-Nova-Api-Version: 2.20 Aug 08 02:32:57.803210 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Id: default Aug 08 02:32:57.803210 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Name: Default Aug 08 02:32:57.803210 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Id: 7caa43bf43ad4e84b4b648e9ab33bd7f Aug 08 02:32:57.803210 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1887360550 Aug 08 02:32:57.803210 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Role: reader,member Aug 08 02:32:57.803210 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Roles: reader,member Aug 08 02:32:57.804785 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/7caa43bf43ad4e84b4b648e9ab33bd7f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/7caa43bf43ad4e84b4b648e9ab33bd7f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_7caa43bf43ad4e84b4b648e9ab33bd7f", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/7caa43bf43ad4e84b4b648e9ab33bd7f"}]}] Aug 08 02:32:57.804785 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1887360550 Aug 08 02:32:57.804785 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Id: 7caa43bf43ad4e84b4b648e9ab33bd7f Aug 08 02:32:57.804785 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1887360550 Aug 08 02:32:57.804785 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User: tempest-AttachVolumeShelveTestJSON-1887360550-project Aug 08 02:32:57.804785 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Id: default Aug 08 02:32:57.804785 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Name: Default Aug 08 02:32:57.804785 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Id: d451fb2f26ce4f50b010fb2a387c4bf3 Aug 08 02:32:57.804785 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1887360550-project Aug 08 02:32:57.804785 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: method: > Aug 08 02:32:57.804785 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: exception: Instance 946fd571-bd98-44f9-89f4-52f0116b642d could not be found. Aug 08 02:32:57.804785 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: action_args: {'id': '946fd571-bd98-44f9-89f4-52f0116b642d'} Aug 08 02:32:57.808340 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:32:57.808340 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:57.808340 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 08 02:32:57.808340 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 08 02:32:57.808340 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 08 02:32:57.808340 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 08 02:32:57.808340 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 08 02:32:57.808340 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 08 02:32:57.808340 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 08 02:32:57.808340 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 08 02:32:57.808340 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 946fd571-bd98-44f9-89f4-52f0116b642d could not be found. Aug 08 02:32:57.808340 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:57.808340 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:32:57.808340 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:57.808340 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:57.808340 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:32:57.808340 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:32:57.808340 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:32:57.808340 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:32:57.808340 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:32:57.810346 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:32:57.810346 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 08 02:32:57.810346 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 08 02:32:57.810346 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 08 02:32:57.810346 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 08 02:32:57.810346 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 08 02:32:57.810346 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 08 02:32:57.810346 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 08 02:32:57.810346 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:32:57.810346 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 946fd571-bd98-44f9-89f4-52f0116b642d could not be found. Aug 08 02:32:57.810346 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:57.810346 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:32:57.810346 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:57.810346 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:57.810346 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:32:57.810346 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:32:57.810346 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:32:57.810346 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:32:57.810346 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 946fd571-bd98-44f9-89f4-52f0116b642d could not be found. Aug 08 02:32:57.810346 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:57.811804 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-67823d39-97cb-4428-9f98-56507d075739 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Returning 404 to user: Instance 946fd571-bd98-44f9-89f4-52f0116b642d could not be found. {{(pid=118352) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:32:57.811804 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-67823d39-97cb-4428-9f98-56507d075739 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "DELETE /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 404 len: 111 microversion: 2.20 time: 0.030979 Aug 08 02:32:57.811804 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 472/934] 10.222.0.27 () {60 vars in 1280 bytes} [Mon Aug 8 02:32:57 2022] DELETE /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Aug 08 02:32:57.817221 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-fb5c3f29-9d2d-4368-80a3-405dc2176e61 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:57.817832 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-fb5c3f29-9d2d-4368-80a3-405dc2176e61 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: 946fd571-bd98-44f9-89f4-52f0116b642d] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:57.823919 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-fb5c3f29-9d2d-4368-80a3-405dc2176e61 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:57.824403 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-fb5c3f29-9d2d-4368-80a3-405dc2176e61 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:57.824861 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-fb5c3f29-9d2d-4368-80a3-405dc2176e61 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:57.825540 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-fb5c3f29-9d2d-4368-80a3-405dc2176e61 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:57.825989 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-fb5c3f29-9d2d-4368-80a3-405dc2176e61 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:57.826406 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-fb5c3f29-9d2d-4368-80a3-405dc2176e61 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:57.839604 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.wsgi [None req-fb5c3f29-9d2d-4368-80a3-405dc2176e61 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] HTTP exception thrown: Instance 946fd571-bd98-44f9-89f4-52f0116b642d could not be found. Aug 08 02:32:57.841380 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-fb5c3f29-9d2d-4368-80a3-405dc2176e61 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Request method failure captured: Aug 08 02:32:57.841380 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: request: GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d HTTP/1.1 Aug 08 02:32:57.841380 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Accept: application/json Aug 08 02:32:57.841380 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Accept-Encoding: identity Aug 08 02:32:57.841380 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Connection: close Aug 08 02:32:57.841380 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Content-Length: 0 Aug 08 02:32:57.841380 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Content-Type: application/json Aug 08 02:32:57.841380 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Host: 10.222.0.27 Aug 08 02:32:57.841380 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Openstack-System-Scope: None Aug 08 02:32:57.841380 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: User-Agent: python-urllib3/1.26.11 Aug 08 02:32:57.841380 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Auth-Token: gAAAAABi8HQZM1u28QkO94W5_BI9_CK6trquSemX1B1L_Z-hB5Xr_ijBdmU9-G9fQkpWPZoqRyWzMs8DIWVoaDURV27YVJYuW-ZoqYszcoIWuLgl1q8SendZ4PZ3TqxVgh6HOFphSeT7fOMCYwtBnDGaMK-ygmFXZVrpyqVNef3pWtqKHJNgdaU Aug 08 02:32:57.841380 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Domain-Id: None Aug 08 02:32:57.841380 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Domain-Name: None Aug 08 02:32:57.841380 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Identity-Status: Confirmed Aug 08 02:32:57.841380 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Is-Admin-Project: True Aug 08 02:32:57.841380 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Openstack-Nova-Api-Version: 2.20 Aug 08 02:32:57.841380 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Domain-Id: default Aug 08 02:32:57.841380 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Domain-Name: Default Aug 08 02:32:57.841380 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Id: 7caa43bf43ad4e84b4b648e9ab33bd7f Aug 08 02:32:57.841380 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1887360550 Aug 08 02:32:57.841380 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Role: reader,member Aug 08 02:32:57.841380 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Roles: reader,member Aug 08 02:32:57.847067 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/7caa43bf43ad4e84b4b648e9ab33bd7f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/7caa43bf43ad4e84b4b648e9ab33bd7f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_7caa43bf43ad4e84b4b648e9ab33bd7f", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/7caa43bf43ad4e84b4b648e9ab33bd7f"}]}] Aug 08 02:32:57.847067 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1887360550 Aug 08 02:32:57.847067 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant-Id: 7caa43bf43ad4e84b4b648e9ab33bd7f Aug 08 02:32:57.847067 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1887360550 Aug 08 02:32:57.847067 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User: tempest-AttachVolumeShelveTestJSON-1887360550-project Aug 08 02:32:57.847067 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Domain-Id: default Aug 08 02:32:57.847067 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Domain-Name: Default Aug 08 02:32:57.847067 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Id: d451fb2f26ce4f50b010fb2a387c4bf3 Aug 08 02:32:57.847067 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1887360550-project Aug 08 02:32:57.847067 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: method: > Aug 08 02:32:57.847067 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: exception: Instance 946fd571-bd98-44f9-89f4-52f0116b642d could not be found. Aug 08 02:32:57.847067 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: action_args: {'id': '946fd571-bd98-44f9-89f4-52f0116b642d'} Aug 08 02:32:57.847863 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:32:57.847863 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:57.847863 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 08 02:32:57.847863 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 08 02:32:57.847863 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 08 02:32:57.847863 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 08 02:32:57.847863 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 08 02:32:57.847863 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 08 02:32:57.847863 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 08 02:32:57.847863 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 08 02:32:57.847863 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 946fd571-bd98-44f9-89f4-52f0116b642d could not be found. Aug 08 02:32:57.847863 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:32:57.847863 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:32:57.847863 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:32:57.847863 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:57.847863 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:32:57.847863 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:32:57.847863 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:32:57.847863 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:32:57.847863 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:32:57.849350 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:32:57.849350 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 08 02:32:57.849350 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 08 02:32:57.849350 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 08 02:32:57.849350 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 08 02:32:57.849350 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 08 02:32:57.849350 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:32:57.849350 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 946fd571-bd98-44f9-89f4-52f0116b642d could not be found. Aug 08 02:32:57.849350 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:32:57.849350 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:32:57.849350 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:32:57.849350 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:57.849350 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:32:57.849350 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:32:57.849350 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:32:57.849350 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:32:57.849350 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 946fd571-bd98-44f9-89f4-52f0116b642d could not be found. Aug 08 02:32:57.849350 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:32:57.850846 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-fb5c3f29-9d2d-4368-80a3-405dc2176e61 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Returning 404 to user: Instance 946fd571-bd98-44f9-89f4-52f0116b642d could not be found. {{(pid=118351) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:32:57.850846 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-fb5c3f29-9d2d-4368-80a3-405dc2176e61 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d" status: 404 len: 111 microversion: 2.20 time: 0.029396 Aug 08 02:32:57.850846 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 463/935] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:32:57 2022] GET /compute/v2.1/servers/946fd571-bd98-44f9-89f4-52f0116b642d => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Aug 08 02:32:58.446426 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-4c4a4800-dfc5-4127-81d0-3ebd738c31a2 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:58.447215 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-4c4a4800-dfc5-4127-81d0-3ebd738c31a2 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] [instance: 209cb22c-fd1c-40f7-b091-43ef7195eabd] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:58.453966 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4c4a4800-dfc5-4127-81d0-3ebd738c31a2 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:58.454455 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4c4a4800-dfc5-4127-81d0-3ebd738c31a2 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:58.454895 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4c4a4800-dfc5-4127-81d0-3ebd738c31a2 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:58.455583 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4c4a4800-dfc5-4127-81d0-3ebd738c31a2 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:58.456036 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4c4a4800-dfc5-4127-81d0-3ebd738c31a2 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:58.456460 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4c4a4800-dfc5-4127-81d0-3ebd738c31a2 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:58.589319 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-4c4a4800-dfc5-4127-81d0-3ebd738c31a2 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ffcf7b8ddd454d9fa3b32f86906cedd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '414b6a313558425cbd3ed5c6da5405a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:58.597207 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-4c4a4800-dfc5-4127-81d0-3ebd738c31a2 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] 10.222.0.27 "GET /compute/v2.1/servers/209cb22c-fd1c-40f7-b091-43ef7195eabd" status: 200 len: 1314 microversion: 2.1 time: 0.154567 Aug 08 02:32:58.597781 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 473/936] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:58 2022] GET /compute/v2.1/servers/209cb22c-fd1c-40f7-b091-43ef7195eabd => generated 1314 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:32:59.148547 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-0f5283e8-3d37-48f0-a99d-56cc5be99e18 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:59.149067 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-0f5283e8-3d37-48f0-a99d-56cc5be99e18 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:59.156439 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0f5283e8-3d37-48f0-a99d-56cc5be99e18 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:59.156950 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0f5283e8-3d37-48f0-a99d-56cc5be99e18 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:59.157386 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0f5283e8-3d37-48f0-a99d-56cc5be99e18 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:59.158069 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0f5283e8-3d37-48f0-a99d-56cc5be99e18 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:59.158524 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0f5283e8-3d37-48f0-a99d-56cc5be99e18 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:59.158955 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0f5283e8-3d37-48f0-a99d-56cc5be99e18 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:59.172925 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.wsgi [None req-0f5283e8-3d37-48f0-a99d-56cc5be99e18 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] HTTP exception thrown: Instance f51a5900-5abc-4188-bbc4-33b0f0f7d091 could not be found. Aug 08 02:32:59.174501 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-0f5283e8-3d37-48f0-a99d-56cc5be99e18 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Request method failure captured: Aug 08 02:32:59.174501 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: request: DELETE /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 HTTP/1.1 Aug 08 02:32:59.174501 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Accept: application/json Aug 08 02:32:59.174501 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Accept-Encoding: identity Aug 08 02:32:59.174501 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Connection: close Aug 08 02:32:59.174501 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Content-Length: 0 Aug 08 02:32:59.174501 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Content-Type: application/json Aug 08 02:32:59.174501 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Host: 10.222.0.27 Aug 08 02:32:59.174501 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Openstack-System-Scope: None Aug 08 02:32:59.174501 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: User-Agent: python-urllib3/1.26.11 Aug 08 02:32:59.174501 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Auth-Token: gAAAAABi8HQZM1u28QkO94W5_BI9_CK6trquSemX1B1L_Z-hB5Xr_ijBdmU9-G9fQkpWPZoqRyWzMs8DIWVoaDURV27YVJYuW-ZoqYszcoIWuLgl1q8SendZ4PZ3TqxVgh6HOFphSeT7fOMCYwtBnDGaMK-ygmFXZVrpyqVNef3pWtqKHJNgdaU Aug 08 02:32:59.174501 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Domain-Id: None Aug 08 02:32:59.174501 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Domain-Name: None Aug 08 02:32:59.174501 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Identity-Status: Confirmed Aug 08 02:32:59.174501 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Is-Admin-Project: True Aug 08 02:32:59.174501 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Openstack-Nova-Api-Version: 2.20 Aug 08 02:32:59.174501 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Domain-Id: default Aug 08 02:32:59.174501 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Domain-Name: Default Aug 08 02:32:59.174501 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Id: 7caa43bf43ad4e84b4b648e9ab33bd7f Aug 08 02:32:59.174501 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1887360550 Aug 08 02:32:59.174501 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Role: reader,member Aug 08 02:32:59.174501 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Roles: reader,member Aug 08 02:32:59.176177 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/7caa43bf43ad4e84b4b648e9ab33bd7f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/7caa43bf43ad4e84b4b648e9ab33bd7f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_7caa43bf43ad4e84b4b648e9ab33bd7f", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/7caa43bf43ad4e84b4b648e9ab33bd7f"}]}] Aug 08 02:32:59.176177 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1887360550 Aug 08 02:32:59.176177 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant-Id: 7caa43bf43ad4e84b4b648e9ab33bd7f Aug 08 02:32:59.176177 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1887360550 Aug 08 02:32:59.176177 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User: tempest-AttachVolumeShelveTestJSON-1887360550-project Aug 08 02:32:59.176177 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Domain-Id: default Aug 08 02:32:59.176177 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Domain-Name: Default Aug 08 02:32:59.176177 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Id: d451fb2f26ce4f50b010fb2a387c4bf3 Aug 08 02:32:59.176177 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1887360550-project Aug 08 02:32:59.176177 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: method: > Aug 08 02:32:59.176177 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: exception: Instance f51a5900-5abc-4188-bbc4-33b0f0f7d091 could not be found. Aug 08 02:32:59.176177 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: action_args: {'id': 'f51a5900-5abc-4188-bbc4-33b0f0f7d091'} Aug 08 02:32:59.177111 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:32:59.177111 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:59.177111 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 08 02:32:59.177111 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 08 02:32:59.177111 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 08 02:32:59.177111 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 08 02:32:59.177111 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 08 02:32:59.177111 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 08 02:32:59.177111 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 08 02:32:59.177111 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 08 02:32:59.177111 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f51a5900-5abc-4188-bbc4-33b0f0f7d091 could not be found. Aug 08 02:32:59.177111 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:32:59.177111 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:32:59.177111 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:32:59.177111 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:59.177111 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:32:59.177111 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:32:59.177111 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:32:59.177111 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:32:59.177111 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:32:59.178571 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:32:59.178571 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 08 02:32:59.178571 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 08 02:32:59.178571 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 08 02:32:59.178571 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 08 02:32:59.178571 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 08 02:32:59.178571 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 08 02:32:59.178571 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 08 02:32:59.178571 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:32:59.178571 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f51a5900-5abc-4188-bbc4-33b0f0f7d091 could not be found. Aug 08 02:32:59.178571 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:32:59.178571 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:32:59.178571 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:32:59.178571 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:59.178571 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:32:59.178571 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:32:59.178571 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:32:59.178571 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:32:59.178571 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f51a5900-5abc-4188-bbc4-33b0f0f7d091 could not be found. Aug 08 02:32:59.178571 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:32:59.180130 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-0f5283e8-3d37-48f0-a99d-56cc5be99e18 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Returning 404 to user: Instance f51a5900-5abc-4188-bbc4-33b0f0f7d091 could not be found. {{(pid=118351) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:32:59.180130 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-0f5283e8-3d37-48f0-a99d-56cc5be99e18 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "DELETE /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 404 len: 111 microversion: 2.20 time: 0.031862 Aug 08 02:32:59.180130 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 464/937] 10.222.0.27 () {60 vars in 1280 bytes} [Mon Aug 8 02:32:59 2022] DELETE /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Aug 08 02:32:59.184313 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-7c814713-18a2-4388-a1b4-f706f3980ed5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:59.184960 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-7c814713-18a2-4388-a1b4-f706f3980ed5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] [instance: f51a5900-5abc-4188-bbc4-33b0f0f7d091] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:59.190866 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7c814713-18a2-4388-a1b4-f706f3980ed5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:59.191350 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7c814713-18a2-4388-a1b4-f706f3980ed5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:59.191994 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7c814713-18a2-4388-a1b4-f706f3980ed5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:59.192463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7c814713-18a2-4388-a1b4-f706f3980ed5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:59.192978 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7c814713-18a2-4388-a1b4-f706f3980ed5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:59.193413 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7c814713-18a2-4388-a1b4-f706f3980ed5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:59.207182 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.wsgi [None req-7c814713-18a2-4388-a1b4-f706f3980ed5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] HTTP exception thrown: Instance f51a5900-5abc-4188-bbc4-33b0f0f7d091 could not be found. Aug 08 02:32:59.208716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-7c814713-18a2-4388-a1b4-f706f3980ed5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Request method failure captured: Aug 08 02:32:59.208716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: request: GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 HTTP/1.1 Aug 08 02:32:59.208716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept: application/json Aug 08 02:32:59.208716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept-Encoding: identity Aug 08 02:32:59.208716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Connection: close Aug 08 02:32:59.208716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Length: 0 Aug 08 02:32:59.208716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Type: application/json Aug 08 02:32:59.208716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Host: 10.222.0.27 Aug 08 02:32:59.208716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Openstack-System-Scope: None Aug 08 02:32:59.208716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: User-Agent: python-urllib3/1.26.11 Aug 08 02:32:59.208716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Auth-Token: gAAAAABi8HQZM1u28QkO94W5_BI9_CK6trquSemX1B1L_Z-hB5Xr_ijBdmU9-G9fQkpWPZoqRyWzMs8DIWVoaDURV27YVJYuW-ZoqYszcoIWuLgl1q8SendZ4PZ3TqxVgh6HOFphSeT7fOMCYwtBnDGaMK-ygmFXZVrpyqVNef3pWtqKHJNgdaU Aug 08 02:32:59.208716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Id: None Aug 08 02:32:59.208716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Name: None Aug 08 02:32:59.208716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Identity-Status: Confirmed Aug 08 02:32:59.208716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Is-Admin-Project: True Aug 08 02:32:59.208716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Openstack-Nova-Api-Version: 2.20 Aug 08 02:32:59.208716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Id: default Aug 08 02:32:59.208716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Name: Default Aug 08 02:32:59.208716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Id: 7caa43bf43ad4e84b4b648e9ab33bd7f Aug 08 02:32:59.208716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1887360550 Aug 08 02:32:59.208716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Role: reader,member Aug 08 02:32:59.208716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Roles: reader,member Aug 08 02:32:59.210405 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/7caa43bf43ad4e84b4b648e9ab33bd7f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/7caa43bf43ad4e84b4b648e9ab33bd7f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_7caa43bf43ad4e84b4b648e9ab33bd7f", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/7caa43bf43ad4e84b4b648e9ab33bd7f"}]}] Aug 08 02:32:59.210405 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1887360550 Aug 08 02:32:59.210405 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Id: 7caa43bf43ad4e84b4b648e9ab33bd7f Aug 08 02:32:59.210405 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1887360550 Aug 08 02:32:59.210405 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User: tempest-AttachVolumeShelveTestJSON-1887360550-project Aug 08 02:32:59.210405 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Id: default Aug 08 02:32:59.210405 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Name: Default Aug 08 02:32:59.210405 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Id: d451fb2f26ce4f50b010fb2a387c4bf3 Aug 08 02:32:59.210405 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1887360550-project Aug 08 02:32:59.210405 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: method: > Aug 08 02:32:59.210405 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: exception: Instance f51a5900-5abc-4188-bbc4-33b0f0f7d091 could not be found. Aug 08 02:32:59.210405 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: action_args: {'id': 'f51a5900-5abc-4188-bbc4-33b0f0f7d091'} Aug 08 02:32:59.211528 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:32:59.211528 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:59.211528 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 08 02:32:59.211528 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 08 02:32:59.211528 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 08 02:32:59.211528 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 08 02:32:59.211528 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 08 02:32:59.211528 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 08 02:32:59.211528 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 08 02:32:59.211528 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 08 02:32:59.211528 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f51a5900-5abc-4188-bbc4-33b0f0f7d091 could not be found. Aug 08 02:32:59.211528 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:59.211528 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:32:59.211528 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:59.211528 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:59.211528 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:32:59.211528 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:32:59.211528 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:32:59.211528 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:32:59.211528 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:32:59.213052 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:32:59.213052 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 08 02:32:59.213052 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 08 02:32:59.213052 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 08 02:32:59.213052 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 08 02:32:59.213052 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 08 02:32:59.213052 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:32:59.213052 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f51a5900-5abc-4188-bbc4-33b0f0f7d091 could not be found. Aug 08 02:32:59.213052 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:59.213052 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:32:59.213052 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:59.213052 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:32:59.213052 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:32:59.213052 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:32:59.213052 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:32:59.213052 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:32:59.213052 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f51a5900-5abc-4188-bbc4-33b0f0f7d091 could not be found. Aug 08 02:32:59.213052 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:32:59.215331 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-7c814713-18a2-4388-a1b4-f706f3980ed5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] Returning 404 to user: Instance f51a5900-5abc-4188-bbc4-33b0f0f7d091 could not be found. {{(pid=118352) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:32:59.215331 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-7c814713-18a2-4388-a1b4-f706f3980ed5 tempest-AttachVolumeShelveTestJSON-1887360550 tempest-AttachVolumeShelveTestJSON-1887360550-project] 10.222.0.27 "GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091" status: 404 len: 111 microversion: 2.20 time: 0.029630 Aug 08 02:32:59.215331 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 474/938] 10.222.0.27 () {60 vars in 1277 bytes} [Mon Aug 8 02:32:59 2022] GET /compute/v2.1/servers/f51a5900-5abc-4188-bbc4-33b0f0f7d091 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Aug 08 02:32:59.618453 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-16a4a83f-bd44-44eb-921f-dce372e51818 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:32:59.623185 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-16a4a83f-bd44-44eb-921f-dce372e51818 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] [instance: 209cb22c-fd1c-40f7-b091-43ef7195eabd] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:32:59.626281 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-16a4a83f-bd44-44eb-921f-dce372e51818 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:59.626951 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-16a4a83f-bd44-44eb-921f-dce372e51818 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:59.627590 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-16a4a83f-bd44-44eb-921f-dce372e51818 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:59.628162 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-16a4a83f-bd44-44eb-921f-dce372e51818 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:32:59.628615 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-16a4a83f-bd44-44eb-921f-dce372e51818 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:32:59.629469 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-16a4a83f-bd44-44eb-921f-dce372e51818 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:32:59.773169 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-16a4a83f-bd44-44eb-921f-dce372e51818 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ffcf7b8ddd454d9fa3b32f86906cedd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '414b6a313558425cbd3ed5c6da5405a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:32:59.780790 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-16a4a83f-bd44-44eb-921f-dce372e51818 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] 10.222.0.27 "GET /compute/v2.1/servers/209cb22c-fd1c-40f7-b091-43ef7195eabd" status: 200 len: 1499 microversion: 2.1 time: 0.166216 Aug 08 02:32:59.781359 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 465/939] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:32:59 2022] GET /compute/v2.1/servers/209cb22c-fd1c-40f7-b091-43ef7195eabd => generated 1499 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:33:00.759797 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-a0fb306a-65d2-42e6-a3cb-2b92d5ac1ac4 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:33:00.760391 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-a0fb306a-65d2-42e6-a3cb-2b92d5ac1ac4 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] [instance: 209cb22c-fd1c-40f7-b091-43ef7195eabd] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:33:00.767739 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a0fb306a-65d2-42e6-a3cb-2b92d5ac1ac4 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:00.768312 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a0fb306a-65d2-42e6-a3cb-2b92d5ac1ac4 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:00.769005 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a0fb306a-65d2-42e6-a3cb-2b92d5ac1ac4 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:00.769789 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a0fb306a-65d2-42e6-a3cb-2b92d5ac1ac4 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:00.770244 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a0fb306a-65d2-42e6-a3cb-2b92d5ac1ac4 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:00.770672 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a0fb306a-65d2-42e6-a3cb-2b92d5ac1ac4 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:00.796897 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-a0fb306a-65d2-42e6-a3cb-2b92d5ac1ac4 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] [instance: 209cb22c-fd1c-40f7-b091-43ef7195eabd] Going to try to terminate instance {{(pid=118352) delete /opt/stack/nova/nova/compute/api.py:2666}} Aug 08 02:33:01.102825 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-a0fb306a-65d2-42e6-a3cb-2b92d5ac1ac4 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] 10.222.0.27 "DELETE /compute/v2.1/servers/209cb22c-fd1c-40f7-b091-43ef7195eabd" status: 204 len: 0 microversion: 2.1 time: 0.347568 Aug 08 02:33:01.103862 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 475/940] 10.222.0.27 () {58 vars in 1239 bytes} [Mon Aug 8 02:33:00 2022] DELETE /compute/v2.1/servers/209cb22c-fd1c-40f7-b091-43ef7195eabd => generated 0 bytes in 349 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 08 02:33:01.112045 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-07f0e468-a4c1-443d-b619-04675fdbce5c tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:33:01.112645 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-07f0e468-a4c1-443d-b619-04675fdbce5c tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] [instance: 209cb22c-fd1c-40f7-b091-43ef7195eabd] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:33:01.119209 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-07f0e468-a4c1-443d-b619-04675fdbce5c tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:01.119696 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-07f0e468-a4c1-443d-b619-04675fdbce5c tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:01.120136 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-07f0e468-a4c1-443d-b619-04675fdbce5c tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:01.121403 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-07f0e468-a4c1-443d-b619-04675fdbce5c tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:01.121858 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-07f0e468-a4c1-443d-b619-04675fdbce5c tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:01.122279 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-07f0e468-a4c1-443d-b619-04675fdbce5c tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:01.298388 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-07f0e468-a4c1-443d-b619-04675fdbce5c tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ffcf7b8ddd454d9fa3b32f86906cedd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '414b6a313558425cbd3ed5c6da5405a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:33:01.311644 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-07f0e468-a4c1-443d-b619-04675fdbce5c tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] 10.222.0.27 "GET /compute/v2.1/servers/209cb22c-fd1c-40f7-b091-43ef7195eabd" status: 200 len: 1505 microversion: 2.1 time: 0.203120 Aug 08 02:33:01.313485 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 466/941] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:33:01 2022] GET /compute/v2.1/servers/209cb22c-fd1c-40f7-b091-43ef7195eabd => generated 1505 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:33:02.335460 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-f6170fa6-9826-469d-871f-97199f8020c5 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:33:02.336810 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-f6170fa6-9826-469d-871f-97199f8020c5 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] [instance: 209cb22c-fd1c-40f7-b091-43ef7195eabd] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:33:02.354212 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f6170fa6-9826-469d-871f-97199f8020c5 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:02.354906 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f6170fa6-9826-469d-871f-97199f8020c5 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:02.356284 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f6170fa6-9826-469d-871f-97199f8020c5 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:02.357351 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f6170fa6-9826-469d-871f-97199f8020c5 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:02.358013 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f6170fa6-9826-469d-871f-97199f8020c5 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:02.358610 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f6170fa6-9826-469d-871f-97199f8020c5 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:02.577039 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-f6170fa6-9826-469d-871f-97199f8020c5 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ffcf7b8ddd454d9fa3b32f86906cedd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '414b6a313558425cbd3ed5c6da5405a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:33:02.617731 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-f6170fa6-9826-469d-871f-97199f8020c5 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] 10.222.0.27 "GET /compute/v2.1/servers/209cb22c-fd1c-40f7-b091-43ef7195eabd" status: 200 len: 1505 microversion: 2.1 time: 0.286616 Aug 08 02:33:02.624059 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 476/942] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:33:02 2022] GET /compute/v2.1/servers/209cb22c-fd1c-40f7-b091-43ef7195eabd => generated 1505 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:33:03.463392 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [req-b255c695-0495-4aba-862c-ebc26440ef84 req-8ce4a593-6899-4b82-a69d-ad28463a1c23 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "209cb22c-fd1c-40f7-b091-43ef7195eabd", "name": "network-vif-deleted", "tag": "706064e3-1136-4ec7-9c05-e6e912a96ad2"}]} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:33:03.473905 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-b255c695-0495-4aba-862c-ebc26440ef84 req-8ce4a593-6899-4b82-a69d-ad28463a1c23 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:03.474377 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-b255c695-0495-4aba-862c-ebc26440ef84 req-8ce4a593-6899-4b82-a69d-ad28463a1c23 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:03.474795 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-b255c695-0495-4aba-862c-ebc26440ef84 req-8ce4a593-6899-4b82-a69d-ad28463a1c23 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:03.487189 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.compute.server_external_events [req-b255c695-0495-4aba-862c-ebc26440ef84 req-8ce4a593-6899-4b82-a69d-ad28463a1c23 service nova] Creating event network-vif-deleted:706064e3-1136-4ec7-9c05-e6e912a96ad2 for instance 209cb22c-fd1c-40f7-b091-43ef7195eabd on hv-cinder-74014 Aug 08 02:33:03.492119 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [req-b255c695-0495-4aba-862c-ebc26440ef84 req-8ce4a593-6899-4b82-a69d-ad28463a1c23 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032551 Aug 08 02:33:03.492906 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 467/943] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:33:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:33:03.646045 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-76219e1c-6528-40ac-b86f-95187d972128 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:33:03.646654 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-76219e1c-6528-40ac-b86f-95187d972128 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] [instance: 209cb22c-fd1c-40f7-b091-43ef7195eabd] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:33:03.653114 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-76219e1c-6528-40ac-b86f-95187d972128 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:03.653596 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-76219e1c-6528-40ac-b86f-95187d972128 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:03.654184 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-76219e1c-6528-40ac-b86f-95187d972128 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:03.660799 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-76219e1c-6528-40ac-b86f-95187d972128 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:03.660799 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-76219e1c-6528-40ac-b86f-95187d972128 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:03.660799 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-76219e1c-6528-40ac-b86f-95187d972128 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:03.718328 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-76219e1c-6528-40ac-b86f-95187d972128 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ffcf7b8ddd454d9fa3b32f86906cedd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '414b6a313558425cbd3ed5c6da5405a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:33:03.727698 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-76219e1c-6528-40ac-b86f-95187d972128 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] 10.222.0.27 "GET /compute/v2.1/servers/209cb22c-fd1c-40f7-b091-43ef7195eabd" status: 200 len: 1299 microversion: 2.1 time: 0.085409 Aug 08 02:33:03.728313 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 477/944] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:33:03 2022] GET /compute/v2.1/servers/209cb22c-fd1c-40f7-b091-43ef7195eabd => generated 1299 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:33:04.750946 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-eeb2e1f3-6a46-493c-a145-8e60e8d0738d tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:33:04.751619 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-eeb2e1f3-6a46-493c-a145-8e60e8d0738d tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] [instance: 209cb22c-fd1c-40f7-b091-43ef7195eabd] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:33:04.758178 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-eeb2e1f3-6a46-493c-a145-8e60e8d0738d tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:04.758865 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-eeb2e1f3-6a46-493c-a145-8e60e8d0738d tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:04.759308 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-eeb2e1f3-6a46-493c-a145-8e60e8d0738d tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:04.760007 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-eeb2e1f3-6a46-493c-a145-8e60e8d0738d tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:04.760544 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-eeb2e1f3-6a46-493c-a145-8e60e8d0738d tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:04.761034 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-eeb2e1f3-6a46-493c-a145-8e60e8d0738d tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:04.784259 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.wsgi [None req-eeb2e1f3-6a46-493c-a145-8e60e8d0738d tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] HTTP exception thrown: Instance 209cb22c-fd1c-40f7-b091-43ef7195eabd could not be found. Aug 08 02:33:04.786000 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-eeb2e1f3-6a46-493c-a145-8e60e8d0738d tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Request method failure captured: Aug 08 02:33:04.786000 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: request: GET /compute/v2.1/servers/209cb22c-fd1c-40f7-b091-43ef7195eabd HTTP/1.1 Aug 08 02:33:04.786000 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Accept: application/json Aug 08 02:33:04.786000 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Accept-Encoding: identity Aug 08 02:33:04.786000 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Connection: close Aug 08 02:33:04.786000 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Content-Length: 0 Aug 08 02:33:04.786000 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Content-Type: application/json Aug 08 02:33:04.786000 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Host: 10.222.0.27 Aug 08 02:33:04.786000 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Openstack-System-Scope: None Aug 08 02:33:04.786000 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: User-Agent: python-urllib3/1.26.11 Aug 08 02:33:04.786000 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Auth-Token: gAAAAABi8HXKG9SYsmkm0nFiOTijFlHCIRv3KiimkAJPRCn3zSSjTRVU4YMV7t3L8udBlfiR-QVKW-AfUNzld8CiOIZpvF3noYv9Le5N6YNq-PtySVD_6RhIYHhQaxBgQBLl96TeJ2Hgc_sJJrh2dbvcaET5TK2rG4EkHkmLZgUdbSwuWPFeoz8 Aug 08 02:33:04.786000 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Domain-Id: None Aug 08 02:33:04.786000 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Domain-Name: None Aug 08 02:33:04.786000 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Identity-Status: Confirmed Aug 08 02:33:04.786000 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Is-Admin-Project: True Aug 08 02:33:04.786000 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Domain-Id: default Aug 08 02:33:04.786000 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Domain-Name: Default Aug 08 02:33:04.786000 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Id: 414b6a313558425cbd3ed5c6da5405a9 Aug 08 02:33:04.786000 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Name: tempest-VolumesActionsTest-1042062957 Aug 08 02:33:04.786000 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Role: reader,member Aug 08 02:33:04.786000 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Roles: reader,member Aug 08 02:33:04.787611 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/414b6a313558425cbd3ed5c6da5405a9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/414b6a313558425cbd3ed5c6da5405a9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_414b6a313558425cbd3ed5c6da5405a9", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/414b6a313558425cbd3ed5c6da5405a9"}]}] Aug 08 02:33:04.787611 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant: tempest-VolumesActionsTest-1042062957 Aug 08 02:33:04.787611 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant-Id: 414b6a313558425cbd3ed5c6da5405a9 Aug 08 02:33:04.787611 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant-Name: tempest-VolumesActionsTest-1042062957 Aug 08 02:33:04.787611 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User: tempest-VolumesActionsTest-1042062957-project Aug 08 02:33:04.787611 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Domain-Id: default Aug 08 02:33:04.787611 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Domain-Name: Default Aug 08 02:33:04.787611 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Id: ffcf7b8ddd454d9fa3b32f86906cedd5 Aug 08 02:33:04.787611 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Name: tempest-VolumesActionsTest-1042062957-project Aug 08 02:33:04.787611 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: method: > Aug 08 02:33:04.787611 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: exception: Instance 209cb22c-fd1c-40f7-b091-43ef7195eabd could not be found. Aug 08 02:33:04.787611 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: action_args: {'id': '209cb22c-fd1c-40f7-b091-43ef7195eabd'} Aug 08 02:33:04.797045 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:33:04.797045 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:33:04.797045 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 08 02:33:04.797045 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 08 02:33:04.797045 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 08 02:33:04.797045 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 08 02:33:04.797045 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 08 02:33:04.797045 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 08 02:33:04.797045 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 08 02:33:04.797045 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 08 02:33:04.797045 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 209cb22c-fd1c-40f7-b091-43ef7195eabd could not be found. Aug 08 02:33:04.797045 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:33:04.797045 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:33:04.797045 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:33:04.797045 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:33:04.797045 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:33:04.797045 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:33:04.797045 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:33:04.797045 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:33:04.797045 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:33:04.798505 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:33:04.798505 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 08 02:33:04.798505 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 08 02:33:04.798505 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 08 02:33:04.798505 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 08 02:33:04.798505 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 08 02:33:04.798505 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:33:04.798505 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 209cb22c-fd1c-40f7-b091-43ef7195eabd could not be found. Aug 08 02:33:04.798505 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:33:04.798505 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:33:04.798505 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:33:04.798505 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:33:04.798505 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:33:04.798505 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:33:04.798505 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:33:04.798505 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:33:04.798505 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 209cb22c-fd1c-40f7-b091-43ef7195eabd could not be found. Aug 08 02:33:04.798505 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:33:04.800144 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-eeb2e1f3-6a46-493c-a145-8e60e8d0738d tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Returning 404 to user: Instance 209cb22c-fd1c-40f7-b091-43ef7195eabd could not be found. {{(pid=118351) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:33:04.800144 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-eeb2e1f3-6a46-493c-a145-8e60e8d0738d tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] 10.222.0.27 "GET /compute/v2.1/servers/209cb22c-fd1c-40f7-b091-43ef7195eabd" status: 404 len: 111 microversion: 2.1 time: 0.042007 Aug 08 02:33:04.800144 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 468/945] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:33:04 2022] GET /compute/v2.1/servers/209cb22c-fd1c-40f7-b091-43ef7195eabd => generated 111 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 08 02:33:04.929113 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-768a8f45-f61d-4e30-84d8-282c7bf65ab4 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-907047918", "imageRef": "30833f47-5987-41ca-b5ee-ad5ecafbb265", "flavorRef": "452", "networks": [{"uuid": "d9455505-5caf-4786-81f8-4735ac0e318e"}]}} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:33:04.988130 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.network.neutron [None req-768a8f45-f61d-4e30-84d8-282c7bf65ab4 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] validate_networks() for [('d9455505-5caf-4786-81f8-4735ac0e318e', None, None, None, None, None)] {{(pid=118352) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 08 02:33:05.556477 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-768a8f45-f61d-4e30-84d8-282c7bf65ab4 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Flavor limits 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 08 02:33:05.556946 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-768a8f45-f61d-4e30-84d8-282c7bf65ab4 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Image limits 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 08 02:33:05.557428 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-768a8f45-f61d-4e30-84d8-282c7bf65ab4 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Flavor pref 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 08 02:33:05.557844 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-768a8f45-f61d-4e30-84d8-282c7bf65ab4 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Image pref 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 08 02:33:05.558264 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-768a8f45-f61d-4e30-84d8-282c7bf65ab4 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 08 02:33:05.567384 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-768a8f45-f61d-4e30-84d8-282c7bf65ab4 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:05.567868 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-768a8f45-f61d-4e30-84d8-282c7bf65ab4 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:05.568353 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-768a8f45-f61d-4e30-84d8-282c7bf65ab4 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:05.569114 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-768a8f45-f61d-4e30-84d8-282c7bf65ab4 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:05.569574 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-768a8f45-f61d-4e30-84d8-282c7bf65ab4 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:05.570003 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-768a8f45-f61d-4e30-84d8-282c7bf65ab4 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:05.600402 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.quota [None req-768a8f45-f61d-4e30-84d8-282c7bf65ab4 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Getting quotas for project 414b6a313558425cbd3ed5c6da5405a9. Resources: {'ram', 'cores', 'instances'} {{(pid=118352) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 08 02:33:05.604586 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.quota [None req-768a8f45-f61d-4e30-84d8-282c7bf65ab4 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Getting quotas for user ffcf7b8ddd454d9fa3b32f86906cedd5 and project 414b6a313558425cbd3ed5c6da5405a9. Resources: {'ram', 'cores', 'instances'} {{(pid=118352) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 08 02:33:05.616697 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-768a8f45-f61d-4e30-84d8-282c7bf65ab4 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Going to run 1 instances... {{(pid=118352) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Aug 08 02:33:05.618986 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-768a8f45-f61d-4e30-84d8-282c7bf65ab4 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] [instance: d8b00165-4068-46b3-904a-20ca45ed8882] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='30833f47-5987-41ca-b5ee-ad5ecafbb265',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118352) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Aug 08 02:33:05.642283 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-c7e988cf-f8d1-47e3-ac78-1574ed6f91b5 tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] Action: 'create', calling method: .version_select at 0x7efc6f7d1dc0>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesSnapshotsTestJSON-volume-230830945"}} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:33:05.745736 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-768a8f45-f61d-4e30-84d8-282c7bf65ab4 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.820792 Aug 08 02:33:05.748176 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 478/946] 10.222.0.27 () {60 vars in 1147 bytes} [Mon Aug 8 02:33:04 2022] POST /compute/v2.1/servers => generated 380 bytes in 823 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 08 02:33:05.761815 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-dfc2f840-bf3a-4f42-ad48-92efd2b38258 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:33:05.764009 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-dfc2f840-bf3a-4f42-ad48-92efd2b38258 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] [instance: d8b00165-4068-46b3-904a-20ca45ed8882] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:33:05.845866 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-dfc2f840-bf3a-4f42-ad48-92efd2b38258 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ffcf7b8ddd454d9fa3b32f86906cedd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '414b6a313558425cbd3ed5c6da5405a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:33:05.854078 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-dfc2f840-bf3a-4f42-ad48-92efd2b38258 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] 10.222.0.27 "GET /compute/v2.1/servers/d8b00165-4068-46b3-904a-20ca45ed8882" status: 200 len: 1213 microversion: 2.1 time: 0.099837 Aug 08 02:33:05.854638 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 479/947] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:33:05 2022] GET /compute/v2.1/servers/d8b00165-4068-46b3-904a-20ca45ed8882 => generated 1213 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:33:06.351498 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-c7e988cf-f8d1-47e3-ac78-1574ed6f91b5 tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] 10.222.0.27 "POST /compute/v2.1/os-volumes" status: 200 len: 344 microversion: 2.1 time: 0.837704 Aug 08 02:33:06.359389 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 469/948] 10.222.0.27 () {60 vars in 1155 bytes} [Mon Aug 8 02:33:05 2022] POST /compute/v2.1/os-volumes => generated 344 bytes in 839 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Aug 08 02:33:06.365095 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-665a86cb-b4ad-4973-999b-307f5a22c0e1 tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] Calling method '.version_select at 0x7efc74081a60>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:33:06.416407 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-665a86cb-b4ad-4973-999b-307f5a22c0e1 tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/b8281c0f-023d-4904-9fe8-34a52f3a16b4" status: 200 len: 344 microversion: 2.1 time: 0.055171 Aug 08 02:33:06.417016 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 480/949] 10.222.0.27 () {58 vars in 1245 bytes} [Mon Aug 8 02:33:06 2022] GET /compute/v2.1/os-volumes/b8281c0f-023d-4904-9fe8-34a52f3a16b4 => generated 344 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:33:06.876064 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-c1647d55-0e28-4141-be34-0d24c0545212 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:33:06.876702 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-c1647d55-0e28-4141-be34-0d24c0545212 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] [instance: d8b00165-4068-46b3-904a-20ca45ed8882] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:33:06.883390 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c1647d55-0e28-4141-be34-0d24c0545212 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:06.883864 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c1647d55-0e28-4141-be34-0d24c0545212 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:06.884291 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c1647d55-0e28-4141-be34-0d24c0545212 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:06.884995 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c1647d55-0e28-4141-be34-0d24c0545212 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:06.885452 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c1647d55-0e28-4141-be34-0d24c0545212 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:06.885877 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c1647d55-0e28-4141-be34-0d24c0545212 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:06.962206 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-c1647d55-0e28-4141-be34-0d24c0545212 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ffcf7b8ddd454d9fa3b32f86906cedd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '414b6a313558425cbd3ed5c6da5405a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:33:06.970386 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-c1647d55-0e28-4141-be34-0d24c0545212 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] 10.222.0.27 "GET /compute/v2.1/servers/d8b00165-4068-46b3-904a-20ca45ed8882" status: 200 len: 1217 microversion: 2.1 time: 0.100607 Aug 08 02:33:06.970938 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 470/950] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:33:06 2022] GET /compute/v2.1/servers/d8b00165-4068-46b3-904a-20ca45ed8882 => generated 1217 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:33:07.431860 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-b3d8112a-9299-46b0-a7fc-f28366d2b209 tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] Calling method '.version_select at 0x7efc7405c1f0>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:33:07.481436 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-b3d8112a-9299-46b0-a7fc-f28366d2b209 tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/b8281c0f-023d-4904-9fe8-34a52f3a16b4" status: 200 len: 345 microversion: 2.1 time: 0.054191 Aug 08 02:33:07.482177 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 481/951] 10.222.0.27 () {58 vars in 1245 bytes} [Mon Aug 8 02:33:07 2022] GET /compute/v2.1/os-volumes/b8281c0f-023d-4904-9fe8-34a52f3a16b4 => generated 345 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:33:07.494712 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-d971b509-cfa2-4027-8595-47dcd132104c tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] Action: 'create', calling method: .version_select at 0x7efc6f7d1af0>, body: {"snapshot": {"volume_id": "b8281c0f-023d-4904-9fe8-34a52f3a16b4", "display_name": "tempest-VolumesSnapshotsTestJSON-Snapshot-1455449391"}} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:33:07.991501 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-479ff6ad-4e1c-4d8c-a066-40fb11d52c6c tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:33:07.992111 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-479ff6ad-4e1c-4d8c-a066-40fb11d52c6c tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] [instance: d8b00165-4068-46b3-904a-20ca45ed8882] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:33:07.999563 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-479ff6ad-4e1c-4d8c-a066-40fb11d52c6c tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:08.000081 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-479ff6ad-4e1c-4d8c-a066-40fb11d52c6c tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:08.000520 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-479ff6ad-4e1c-4d8c-a066-40fb11d52c6c tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:08.001282 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-479ff6ad-4e1c-4d8c-a066-40fb11d52c6c tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:08.001744 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-479ff6ad-4e1c-4d8c-a066-40fb11d52c6c tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:08.002171 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-479ff6ad-4e1c-4d8c-a066-40fb11d52c6c tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:08.026421 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-d971b509-cfa2-4027-8595-47dcd132104c tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] 10.222.0.27 "POST /compute/v2.1/os-snapshots" status: 200 len: 287 microversion: 2.1 time: 0.535586 Aug 08 02:33:08.026911 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 471/952] 10.222.0.27 () {60 vars in 1162 bytes} [Mon Aug 8 02:33:07 2022] POST /compute/v2.1/os-snapshots => generated 287 bytes in 537 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 08 02:33:08.037485 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-752c9e96-ee97-4f4d-98d7-0cb49b12f3f4 tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] Calling method '.version_select at 0x7efc6f7dedc0>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:33:08.084865 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-479ff6ad-4e1c-4d8c-a066-40fb11d52c6c tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ffcf7b8ddd454d9fa3b32f86906cedd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '414b6a313558425cbd3ed5c6da5405a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:33:08.105083 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-479ff6ad-4e1c-4d8c-a066-40fb11d52c6c tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] 10.222.0.27 "GET /compute/v2.1/servers/d8b00165-4068-46b3-904a-20ca45ed8882" status: 200 len: 1265 microversion: 2.1 time: 0.105440 Aug 08 02:33:08.105083 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 482/953] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:33:07 2022] GET /compute/v2.1/servers/d8b00165-4068-46b3-904a-20ca45ed8882 => generated 1265 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:33:08.105245 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-752c9e96-ee97-4f4d-98d7-0cb49b12f3f4 tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] 10.222.0.27 "GET /compute/v2.1/os-snapshots/418fc582-eb87-4081-ba27-e37ffb085b85" status: 200 len: 287 microversion: 2.1 time: 0.061220 Aug 08 02:33:08.105245 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 472/954] 10.222.0.27 () {58 vars in 1251 bytes} [Mon Aug 8 02:33:08 2022] GET /compute/v2.1/os-snapshots/418fc582-eb87-4081-ba27-e37ffb085b85 => generated 287 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 08 02:33:08.105681 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-8fe2d5ee-4e73-4d4f-8c9c-a1854cbe1f5c tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] Calling method '.version_select at 0x7efc7404aaf0>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:33:08.137618 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-8fe2d5ee-4e73-4d4f-8c9c-a1854cbe1f5c tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] 10.222.0.27 "GET /compute/v2.1/os-snapshots" status: 200 len: 290 microversion: 2.1 time: 0.035452 Aug 08 02:33:08.138254 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 483/955] 10.222.0.27 () {58 vars in 1140 bytes} [Mon Aug 8 02:33:08 2022] GET /compute/v2.1/os-snapshots => generated 290 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:33:08.149508 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-6eaa748b-330b-4808-bcf6-dc96a6b8f8e5 tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] Calling method '.version_select at 0x7efc7406a310>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:33:08.189706 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-6eaa748b-330b-4808-bcf6-dc96a6b8f8e5 tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] 10.222.0.27 "GET /compute/v2.1/os-snapshots/418fc582-eb87-4081-ba27-e37ffb085b85" status: 200 len: 287 microversion: 2.1 time: 0.043952 Aug 08 02:33:08.190566 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 473/956] 10.222.0.27 () {58 vars in 1251 bytes} [Mon Aug 8 02:33:08 2022] GET /compute/v2.1/os-snapshots/418fc582-eb87-4081-ba27-e37ffb085b85 => generated 287 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:33:09.114581 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-08bb978c-5ee6-4eec-b215-57215029138d tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:33:09.115197 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-08bb978c-5ee6-4eec-b215-57215029138d tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] [instance: d8b00165-4068-46b3-904a-20ca45ed8882] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:33:09.122447 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-08bb978c-5ee6-4eec-b215-57215029138d tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:09.123047 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-08bb978c-5ee6-4eec-b215-57215029138d tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:09.123443 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-08bb978c-5ee6-4eec-b215-57215029138d tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:09.124142 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-08bb978c-5ee6-4eec-b215-57215029138d tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:09.124575 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-08bb978c-5ee6-4eec-b215-57215029138d tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:09.125050 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-08bb978c-5ee6-4eec-b215-57215029138d tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:09.203805 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-aadc941b-70af-4b45-be5f-fa58a80c5892 tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] Calling method '.version_select at 0x7efc6f7ea670>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:33:09.234133 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-aadc941b-70af-4b45-be5f-fa58a80c5892 tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] 10.222.0.27 "GET /compute/v2.1/os-snapshots/418fc582-eb87-4081-ba27-e37ffb085b85" status: 200 len: 287 microversion: 2.1 time: 0.035174 Aug 08 02:33:09.234711 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 474/957] 10.222.0.27 () {58 vars in 1251 bytes} [Mon Aug 8 02:33:09 2022] GET /compute/v2.1/os-snapshots/418fc582-eb87-4081-ba27-e37ffb085b85 => generated 287 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 08 02:33:09.254243 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-08bb978c-5ee6-4eec-b215-57215029138d tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ffcf7b8ddd454d9fa3b32f86906cedd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '414b6a313558425cbd3ed5c6da5405a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:33:09.261678 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-08bb978c-5ee6-4eec-b215-57215029138d tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] 10.222.0.27 "GET /compute/v2.1/servers/d8b00165-4068-46b3-904a-20ca45ed8882" status: 200 len: 1313 microversion: 2.1 time: 0.151597 Aug 08 02:33:09.262244 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 484/958] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:33:09 2022] GET /compute/v2.1/servers/d8b00165-4068-46b3-904a-20ca45ed8882 => generated 1313 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:33:10.247051 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-a51f5480-c0a3-42ca-8769-3ccf07369d7b tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] Calling method '.version_select at 0x7efc6f7d1dc0>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:33:10.277321 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-a51f5480-c0a3-42ca-8769-3ccf07369d7b tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] 10.222.0.27 "GET /compute/v2.1/os-snapshots/418fc582-eb87-4081-ba27-e37ffb085b85" status: 200 len: 287 microversion: 2.1 time: 0.034662 Aug 08 02:33:10.278053 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 475/959] 10.222.0.27 () {58 vars in 1251 bytes} [Mon Aug 8 02:33:10 2022] GET /compute/v2.1/os-snapshots/418fc582-eb87-4081-ba27-e37ffb085b85 => generated 287 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:33:10.283962 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-dd18258b-fb71-4f52-9a62-6085e7497d99 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:33:10.284573 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-dd18258b-fb71-4f52-9a62-6085e7497d99 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] [instance: d8b00165-4068-46b3-904a-20ca45ed8882] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:33:10.290934 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dd18258b-fb71-4f52-9a62-6085e7497d99 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:10.291409 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dd18258b-fb71-4f52-9a62-6085e7497d99 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:10.291841 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dd18258b-fb71-4f52-9a62-6085e7497d99 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:10.292519 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dd18258b-fb71-4f52-9a62-6085e7497d99 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:10.293002 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dd18258b-fb71-4f52-9a62-6085e7497d99 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:10.293426 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dd18258b-fb71-4f52-9a62-6085e7497d99 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:10.425772 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-dd18258b-fb71-4f52-9a62-6085e7497d99 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ffcf7b8ddd454d9fa3b32f86906cedd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '414b6a313558425cbd3ed5c6da5405a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:33:10.433406 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-dd18258b-fb71-4f52-9a62-6085e7497d99 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] 10.222.0.27 "GET /compute/v2.1/servers/d8b00165-4068-46b3-904a-20ca45ed8882" status: 200 len: 1313 microversion: 2.1 time: 0.154171 Aug 08 02:33:10.433961 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 485/960] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:33:10 2022] GET /compute/v2.1/servers/d8b00165-4068-46b3-904a-20ca45ed8882 => generated 1313 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:33:11.290233 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-db9e9b44-5aeb-4f81-9c92-851184da12d3 tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] Calling method '.version_select at 0x7efc7406a550>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:33:11.321102 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-db9e9b44-5aeb-4f81-9c92-851184da12d3 tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] 10.222.0.27 "GET /compute/v2.1/os-snapshots/418fc582-eb87-4081-ba27-e37ffb085b85" status: 200 len: 288 microversion: 2.1 time: 0.035235 Aug 08 02:33:11.321648 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 476/961] 10.222.0.27 () {58 vars in 1251 bytes} [Mon Aug 8 02:33:11 2022] GET /compute/v2.1/os-snapshots/418fc582-eb87-4081-ba27-e37ffb085b85 => generated 288 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:33:11.333095 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-a60c88cf-6be2-4c56-a67f-b30edefab415 tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] Calling method '.version_select at 0x7efc740bc040>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:33:11.457333 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-d6c60878-c1ea-444c-ade3-6c85260fc9f6 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:33:11.458255 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-d6c60878-c1ea-444c-ade3-6c85260fc9f6 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] [instance: d8b00165-4068-46b3-904a-20ca45ed8882] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:33:11.458374 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-a60c88cf-6be2-4c56-a67f-b30edefab415 tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] 10.222.0.27 "DELETE /compute/v2.1/os-snapshots/418fc582-eb87-4081-ba27-e37ffb085b85" status: 202 len: 0 microversion: 2.1 time: 0.129263 Aug 08 02:33:11.458890 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 486/962] 10.222.0.27 () {58 vars in 1254 bytes} [Mon Aug 8 02:33:11 2022] DELETE /compute/v2.1/os-snapshots/418fc582-eb87-4081-ba27-e37ffb085b85 => generated 0 bytes in 131 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 08 02:33:11.466277 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d6c60878-c1ea-444c-ade3-6c85260fc9f6 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:11.467802 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-d3bb4046-9844-4f6d-8383-84c8821a16b5 tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] Calling method '.version_select at 0x7efc74081dc0>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:33:11.468437 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d6c60878-c1ea-444c-ade3-6c85260fc9f6 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:11.469068 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d6c60878-c1ea-444c-ade3-6c85260fc9f6 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:11.474023 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d6c60878-c1ea-444c-ade3-6c85260fc9f6 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:11.474023 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d6c60878-c1ea-444c-ade3-6c85260fc9f6 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:11.474023 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d6c60878-c1ea-444c-ade3-6c85260fc9f6 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:11.474023 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell1:SELECT=100 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:11.507771 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-d3bb4046-9844-4f6d-8383-84c8821a16b5 tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] 10.222.0.27 "GET /compute/v2.1/os-snapshots/418fc582-eb87-4081-ba27-e37ffb085b85" status: 200 len: 287 microversion: 2.1 time: 0.044080 Aug 08 02:33:11.508459 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 487/963] 10.222.0.27 () {58 vars in 1251 bytes} [Mon Aug 8 02:33:11 2022] GET /compute/v2.1/os-snapshots/418fc582-eb87-4081-ba27-e37ffb085b85 => generated 287 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 08 02:33:11.610041 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-d6c60878-c1ea-444c-ade3-6c85260fc9f6 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ffcf7b8ddd454d9fa3b32f86906cedd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '414b6a313558425cbd3ed5c6da5405a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:33:11.617947 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-d6c60878-c1ea-444c-ade3-6c85260fc9f6 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] 10.222.0.27 "GET /compute/v2.1/servers/d8b00165-4068-46b3-904a-20ca45ed8882" status: 200 len: 1313 microversion: 2.1 time: 0.168058 Aug 08 02:33:11.618525 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 477/964] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:33:11 2022] GET /compute/v2.1/servers/d8b00165-4068-46b3-904a-20ca45ed8882 => generated 1313 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:33:11.627575 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [req-270d5fec-d8ea-484b-8248-231ecaf057bf req-631bbbc4-ed09-4157-a2bf-060d7f7f5200 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d8b00165-4068-46b3-904a-20ca45ed8882", "tag": "35865bc3-fb2d-47aa-9dca-c85e9076883c"}]} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:33:11.637539 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-270d5fec-d8ea-484b-8248-231ecaf057bf req-631bbbc4-ed09-4157-a2bf-060d7f7f5200 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:11.638013 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-270d5fec-d8ea-484b-8248-231ecaf057bf req-631bbbc4-ed09-4157-a2bf-060d7f7f5200 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:11.638683 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-270d5fec-d8ea-484b-8248-231ecaf057bf req-631bbbc4-ed09-4157-a2bf-060d7f7f5200 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:11.649104 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.compute.server_external_events [req-270d5fec-d8ea-484b-8248-231ecaf057bf req-631bbbc4-ed09-4157-a2bf-060d7f7f5200 service nova] Creating event network-changed:35865bc3-fb2d-47aa-9dca-c85e9076883c for instance d8b00165-4068-46b3-904a-20ca45ed8882 on hv-cinder-74014 Aug 08 02:33:11.654420 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [req-270d5fec-d8ea-484b-8248-231ecaf057bf req-631bbbc4-ed09-4157-a2bf-060d7f7f5200 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030421 Aug 08 02:33:11.656592 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 488/965] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:33:11 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 08 02:33:12.519560 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-c75bdd30-db05-405e-a385-47c7f0823ab2 tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] Calling method '.version_select at 0x7efc740438b0>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:33:12.552021 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-c75bdd30-db05-405e-a385-47c7f0823ab2 tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] 10.222.0.27 "GET /compute/v2.1/os-snapshots/418fc582-eb87-4081-ba27-e37ffb085b85" status: 200 len: 287 microversion: 2.1 time: 0.036234 Aug 08 02:33:12.552590 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 478/966] 10.222.0.27 () {58 vars in 1251 bytes} [Mon Aug 8 02:33:12 2022] GET /compute/v2.1/os-snapshots/418fc582-eb87-4081-ba27-e37ffb085b85 => generated 287 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:33:12.640083 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-db8f4138-0634-4ad3-b54c-f9d7f17e6974 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:33:12.640825 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-db8f4138-0634-4ad3-b54c-f9d7f17e6974 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] [instance: d8b00165-4068-46b3-904a-20ca45ed8882] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:33:12.647191 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-db8f4138-0634-4ad3-b54c-f9d7f17e6974 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:12.647721 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-db8f4138-0634-4ad3-b54c-f9d7f17e6974 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:12.648161 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-db8f4138-0634-4ad3-b54c-f9d7f17e6974 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:12.648901 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-db8f4138-0634-4ad3-b54c-f9d7f17e6974 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:12.649359 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-db8f4138-0634-4ad3-b54c-f9d7f17e6974 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:12.649785 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-db8f4138-0634-4ad3-b54c-f9d7f17e6974 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:12.780749 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-db8f4138-0634-4ad3-b54c-f9d7f17e6974 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ffcf7b8ddd454d9fa3b32f86906cedd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '414b6a313558425cbd3ed5c6da5405a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:33:12.788370 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-db8f4138-0634-4ad3-b54c-f9d7f17e6974 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] 10.222.0.27 "GET /compute/v2.1/servers/d8b00165-4068-46b3-904a-20ca45ed8882" status: 200 len: 1313 microversion: 2.1 time: 0.153272 Aug 08 02:33:12.788998 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 489/967] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:33:12 2022] GET /compute/v2.1/servers/d8b00165-4068-46b3-904a-20ca45ed8882 => generated 1313 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:33:13.566170 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-cc06edab-739b-48f3-8393-f124059c7ac4 tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] Calling method '.version_select at 0x7efc6f7d1790>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:33:13.596353 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-cc06edab-739b-48f3-8393-f124059c7ac4 tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] 10.222.0.27 "GET /compute/v2.1/os-snapshots/418fc582-eb87-4081-ba27-e37ffb085b85" status: 200 len: 287 microversion: 2.1 time: 0.035586 Aug 08 02:33:13.596957 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 479/968] 10.222.0.27 () {58 vars in 1251 bytes} [Mon Aug 8 02:33:13 2022] GET /compute/v2.1/os-snapshots/418fc582-eb87-4081-ba27-e37ffb085b85 => generated 287 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:33:13.809721 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-5a3127d4-a067-4412-9546-6bdda78daeef tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:33:13.810336 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-5a3127d4-a067-4412-9546-6bdda78daeef tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] [instance: d8b00165-4068-46b3-904a-20ca45ed8882] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:33:13.817474 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5a3127d4-a067-4412-9546-6bdda78daeef tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:13.817956 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5a3127d4-a067-4412-9546-6bdda78daeef tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:13.818387 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5a3127d4-a067-4412-9546-6bdda78daeef tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:13.819082 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5a3127d4-a067-4412-9546-6bdda78daeef tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:13.819681 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5a3127d4-a067-4412-9546-6bdda78daeef tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:13.820110 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-5a3127d4-a067-4412-9546-6bdda78daeef tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:13.824482 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell1:INSERT=2,nova_cell1:SELECT=97,nova_cell1:UPDATE=1 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:13.956861 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-5a3127d4-a067-4412-9546-6bdda78daeef tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ffcf7b8ddd454d9fa3b32f86906cedd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '414b6a313558425cbd3ed5c6da5405a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:33:13.964476 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-5a3127d4-a067-4412-9546-6bdda78daeef tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] 10.222.0.27 "GET /compute/v2.1/servers/d8b00165-4068-46b3-904a-20ca45ed8882" status: 200 len: 1313 microversion: 2.1 time: 0.158871 Aug 08 02:33:13.965152 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 490/969] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:33:13 2022] GET /compute/v2.1/servers/d8b00165-4068-46b3-904a-20ca45ed8882 => generated 1313 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:33:14.609566 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-9afb1734-f760-44ef-baf7-efde6f1c30ef tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] Calling method '.version_select at 0x7efc740835e0>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:33:14.638149 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.wsgi [None req-9afb1734-f760-44ef-baf7-efde6f1c30ef tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] HTTP exception thrown: Snapshot 418fc582-eb87-4081-ba27-e37ffb085b85 could not be found. Aug 08 02:33:14.640660 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-9afb1734-f760-44ef-baf7-efde6f1c30ef tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] Request method failure captured: Aug 08 02:33:14.640660 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: request: GET /compute/v2.1/os-snapshots/418fc582-eb87-4081-ba27-e37ffb085b85 HTTP/1.1 Aug 08 02:33:14.640660 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Accept: application/json Aug 08 02:33:14.640660 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Accept-Encoding: identity Aug 08 02:33:14.640660 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Connection: close Aug 08 02:33:14.640660 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Content-Length: 0 Aug 08 02:33:14.640660 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Content-Type: application/json Aug 08 02:33:14.640660 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Host: 10.222.0.27 Aug 08 02:33:14.640660 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Openstack-System-Scope: None Aug 08 02:33:14.640660 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: User-Agent: python-urllib3/1.26.11 Aug 08 02:33:14.640660 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Auth-Token: gAAAAABi8HXhaYxhkLc7eS6ItrppO8DKmi6Y6b2baYkXD4bDhyb85iXbV0QGMMqvoKAed38RxJZ0AdkiY6o33MKBWpg8Cg3O36N5LMUYksSfDFMDpM6ec01Puhd8OfLAZKhPw9P6ReR4M0g9SEDUhnfcdxHwNlgjpBHhYIyZgjyrAbj62eBht8w Aug 08 02:33:14.640660 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Domain-Id: None Aug 08 02:33:14.640660 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Domain-Name: None Aug 08 02:33:14.640660 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Identity-Status: Confirmed Aug 08 02:33:14.640660 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Is-Admin-Project: True Aug 08 02:33:14.640660 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Domain-Id: default Aug 08 02:33:14.640660 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Domain-Name: Default Aug 08 02:33:14.640660 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Id: 962280636320471080abda11086f1e9e Aug 08 02:33:14.640660 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-25755299 Aug 08 02:33:14.640660 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Role: reader,member Aug 08 02:33:14.640660 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Roles: reader,member Aug 08 02:33:14.642931 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/962280636320471080abda11086f1e9e"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/962280636320471080abda11086f1e9e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_962280636320471080abda11086f1e9e", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/962280636320471080abda11086f1e9e"}]}] Aug 08 02:33:14.642931 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant: tempest-VolumesSnapshotsTestJSON-25755299 Aug 08 02:33:14.642931 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant-Id: 962280636320471080abda11086f1e9e Aug 08 02:33:14.642931 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-25755299 Aug 08 02:33:14.642931 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User: tempest-VolumesSnapshotsTestJSON-25755299-project Aug 08 02:33:14.642931 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Domain-Id: default Aug 08 02:33:14.642931 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Domain-Name: Default Aug 08 02:33:14.642931 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Id: 78e6c594dd7f4cd8b20c1361c1eadcc9 Aug 08 02:33:14.642931 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Name: tempest-VolumesSnapshotsTestJSON-25755299-project Aug 08 02:33:14.642931 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: method: Aug 08 02:33:14.642931 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: exception: Snapshot 418fc582-eb87-4081-ba27-e37ffb085b85 could not be found. Aug 08 02:33:14.642931 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: action_args: {'id': '418fc582-eb87-4081-ba27-e37ffb085b85'} Aug 08 02:33:14.642931 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:33:14.644139 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:33:14.644139 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Aug 08 02:33:14.644139 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Aug 08 02:33:14.644139 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Aug 08 02:33:14.644139 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Aug 08 02:33:14.644139 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Aug 08 02:33:14.644139 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Aug 08 02:33:14.644139 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 08 02:33:14.644139 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 08 02:33:14.644139 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 08 02:33:14.644139 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 08 02:33:14.644139 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 08 02:33:14.644139 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 08 02:33:14.644139 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 08 02:33:14.644139 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 08 02:33:14.644139 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Snapshot 418fc582-eb87-4081-ba27-e37ffb085b85 could not be found. (HTTP 404) (Request-ID: req-7db4ab6e-f22f-450e-9049-c696539129d0) Aug 08 02:33:14.644139 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:33:14.644139 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:33:14.645664 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:33:14.645664 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:33:14.645664 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 617, in show Aug 08 02:33:14.645664 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get_snapshot(context, id) Aug 08 02:33:14.645664 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 08 02:33:14.645664 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 08 02:33:14.645664 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper Aug 08 02:33:14.645664 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi _reraise(exception.SnapshotNotFound(snapshot_id=snapshot_id)) Aug 08 02:33:14.645664 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 08 02:33:14.645664 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 08 02:33:14.645664 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Aug 08 02:33:14.645664 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Aug 08 02:33:14.645664 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Aug 08 02:33:14.645664 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Aug 08 02:33:14.645664 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Aug 08 02:33:14.645664 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Aug 08 02:33:14.645664 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 08 02:33:14.645664 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 08 02:33:14.645664 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 08 02:33:14.647095 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 08 02:33:14.647095 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 08 02:33:14.647095 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 08 02:33:14.647095 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 08 02:33:14.647095 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 08 02:33:14.647095 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi nova.exception.SnapshotNotFound: Snapshot 418fc582-eb87-4081-ba27-e37ffb085b85 could not be found. Aug 08 02:33:14.647095 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:33:14.647095 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:33:14.647095 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:33:14.647095 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:33:14.647095 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:33:14.647095 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:33:14.647095 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:33:14.647095 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:33:14.647095 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 08 02:33:14.647095 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 08 02:33:14.647095 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:33:14.647095 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:33:14.647095 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 619, in show Aug 08 02:33:14.647095 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:33:14.648654 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Snapshot 418fc582-eb87-4081-ba27-e37ffb085b85 could not be found. Aug 08 02:33:14.648654 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:33:14.648654 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:33:14.648654 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:33:14.648654 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:33:14.648654 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:33:14.648654 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:33:14.648654 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:33:14.648654 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:33:14.648654 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Snapshot 418fc582-eb87-4081-ba27-e37ffb085b85 could not be found. Aug 08 02:33:14.648654 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:33:14.648654 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-9afb1734-f760-44ef-baf7-efde6f1c30ef tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] Returning 404 to user: Snapshot 418fc582-eb87-4081-ba27-e37ffb085b85 could not be found. {{(pid=118351) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:33:14.648654 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-9afb1734-f760-44ef-baf7-efde6f1c30ef tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] 10.222.0.27 "GET /compute/v2.1/os-snapshots/418fc582-eb87-4081-ba27-e37ffb085b85" status: 404 len: 111 microversion: 2.1 time: 0.038357 Aug 08 02:33:14.648654 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 480/970] 10.222.0.27 () {58 vars in 1251 bytes} [Mon Aug 8 02:33:14 2022] GET /compute/v2.1/os-snapshots/418fc582-eb87-4081-ba27-e37ffb085b85 => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 08 02:33:14.654413 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-958eb8d7-e1eb-47b1-a9a6-1f1d0b117e2a tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] Calling method '.version_select at 0x7efc74077ca0>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:33:14.899645 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-958eb8d7-e1eb-47b1-a9a6-1f1d0b117e2a tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] 10.222.0.27 "DELETE /compute/v2.1/os-volumes/b8281c0f-023d-4904-9fe8-34a52f3a16b4" status: 202 len: 0 microversion: 2.1 time: 0.249785 Aug 08 02:33:14.900436 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 491/971] 10.222.0.27 () {58 vars in 1248 bytes} [Mon Aug 8 02:33:14 2022] DELETE /compute/v2.1/os-volumes/b8281c0f-023d-4904-9fe8-34a52f3a16b4 => generated 0 bytes in 252 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 08 02:33:14.909103 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-c61d6431-3c67-4ef5-b25a-4ae76b48fe0e tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] Calling method '.version_select at 0x7efc74038c10>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:33:14.967487 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-c61d6431-3c67-4ef5-b25a-4ae76b48fe0e tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/b8281c0f-023d-4904-9fe8-34a52f3a16b4" status: 200 len: 344 microversion: 2.1 time: 0.062578 Aug 08 02:33:14.968052 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 481/972] 10.222.0.27 () {58 vars in 1245 bytes} [Mon Aug 8 02:33:14 2022] GET /compute/v2.1/os-volumes/b8281c0f-023d-4904-9fe8-34a52f3a16b4 => generated 344 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:33:14.987243 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-9bb34ce1-c9e0-4408-bdb9-dbca037c39ef tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:33:14.988485 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-9bb34ce1-c9e0-4408-bdb9-dbca037c39ef tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] [instance: d8b00165-4068-46b3-904a-20ca45ed8882] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:33:14.993931 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9bb34ce1-c9e0-4408-bdb9-dbca037c39ef tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:14.994410 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9bb34ce1-c9e0-4408-bdb9-dbca037c39ef tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:14.994846 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9bb34ce1-c9e0-4408-bdb9-dbca037c39ef tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:14.995530 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9bb34ce1-c9e0-4408-bdb9-dbca037c39ef tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:14.995983 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9bb34ce1-c9e0-4408-bdb9-dbca037c39ef tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:14.996403 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9bb34ce1-c9e0-4408-bdb9-dbca037c39ef tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:15.144419 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-9bb34ce1-c9e0-4408-bdb9-dbca037c39ef tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ffcf7b8ddd454d9fa3b32f86906cedd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '414b6a313558425cbd3ed5c6da5405a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:33:15.151785 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-9bb34ce1-c9e0-4408-bdb9-dbca037c39ef tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] 10.222.0.27 "GET /compute/v2.1/servers/d8b00165-4068-46b3-904a-20ca45ed8882" status: 200 len: 1313 microversion: 2.1 time: 0.170477 Aug 08 02:33:15.154810 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 492/973] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:33:14 2022] GET /compute/v2.1/servers/d8b00165-4068-46b3-904a-20ca45ed8882 => generated 1313 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:33:15.981586 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-cb01702f-54d1-49e6-8f63-bfeabe92aba5 tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] Calling method '.version_select at 0x7efc6f7d1af0>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:33:16.009820 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.wsgi [None req-cb01702f-54d1-49e6-8f63-bfeabe92aba5 tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] HTTP exception thrown: Volume b8281c0f-023d-4904-9fe8-34a52f3a16b4 could not be found. Aug 08 02:33:16.011746 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-cb01702f-54d1-49e6-8f63-bfeabe92aba5 tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] Request method failure captured: Aug 08 02:33:16.011746 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: request: GET /compute/v2.1/os-volumes/b8281c0f-023d-4904-9fe8-34a52f3a16b4 HTTP/1.1 Aug 08 02:33:16.011746 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Accept: application/json Aug 08 02:33:16.011746 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Accept-Encoding: identity Aug 08 02:33:16.011746 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Connection: close Aug 08 02:33:16.011746 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Content-Length: 0 Aug 08 02:33:16.011746 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Content-Type: application/json Aug 08 02:33:16.011746 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Host: 10.222.0.27 Aug 08 02:33:16.011746 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Openstack-System-Scope: None Aug 08 02:33:16.011746 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: User-Agent: python-urllib3/1.26.11 Aug 08 02:33:16.011746 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Auth-Token: gAAAAABi8HXhaYxhkLc7eS6ItrppO8DKmi6Y6b2baYkXD4bDhyb85iXbV0QGMMqvoKAed38RxJZ0AdkiY6o33MKBWpg8Cg3O36N5LMUYksSfDFMDpM6ec01Puhd8OfLAZKhPw9P6ReR4M0g9SEDUhnfcdxHwNlgjpBHhYIyZgjyrAbj62eBht8w Aug 08 02:33:16.011746 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Domain-Id: None Aug 08 02:33:16.011746 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Domain-Name: None Aug 08 02:33:16.011746 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Identity-Status: Confirmed Aug 08 02:33:16.011746 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Is-Admin-Project: True Aug 08 02:33:16.011746 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Domain-Id: default Aug 08 02:33:16.011746 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Domain-Name: Default Aug 08 02:33:16.011746 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Id: 962280636320471080abda11086f1e9e Aug 08 02:33:16.011746 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-25755299 Aug 08 02:33:16.011746 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Role: reader,member Aug 08 02:33:16.011746 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Roles: reader,member Aug 08 02:33:16.013374 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/962280636320471080abda11086f1e9e"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/962280636320471080abda11086f1e9e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_962280636320471080abda11086f1e9e", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/962280636320471080abda11086f1e9e"}]}] Aug 08 02:33:16.013374 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant: tempest-VolumesSnapshotsTestJSON-25755299 Aug 08 02:33:16.013374 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant-Id: 962280636320471080abda11086f1e9e Aug 08 02:33:16.013374 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-25755299 Aug 08 02:33:16.013374 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User: tempest-VolumesSnapshotsTestJSON-25755299-project Aug 08 02:33:16.013374 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Domain-Id: default Aug 08 02:33:16.013374 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Domain-Name: Default Aug 08 02:33:16.013374 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Id: 78e6c594dd7f4cd8b20c1361c1eadcc9 Aug 08 02:33:16.013374 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Name: tempest-VolumesSnapshotsTestJSON-25755299-project Aug 08 02:33:16.013374 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: method: Aug 08 02:33:16.013374 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: exception: Volume b8281c0f-023d-4904-9fe8-34a52f3a16b4 could not be found. Aug 08 02:33:16.013374 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: action_args: {'id': 'b8281c0f-023d-4904-9fe8-34a52f3a16b4'} Aug 08 02:33:16.013374 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:33:16.014333 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:33:16.014333 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 08 02:33:16.014333 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 08 02:33:16.014333 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Aug 08 02:33:16.014333 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi item = cinderclient( Aug 08 02:33:16.014333 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Aug 08 02:33:16.014333 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Aug 08 02:33:16.014333 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 08 02:33:16.014333 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 08 02:33:16.014333 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 08 02:33:16.014333 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 08 02:33:16.014333 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 08 02:33:16.014333 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 08 02:33:16.014333 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 08 02:33:16.014333 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 08 02:33:16.014333 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume b8281c0f-023d-4904-9fe8-34a52f3a16b4 could not be found. (HTTP 404) (Request-ID: req-98a82974-2714-48b7-92c9-0dfee37455d9) Aug 08 02:33:16.014333 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:33:16.014333 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:33:16.014333 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:33:16.016322 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:33:16.016322 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Aug 08 02:33:16.016322 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Aug 08 02:33:16.016322 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 08 02:33:16.016322 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 08 02:33:16.016322 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Aug 08 02:33:16.016322 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Aug 08 02:33:16.016322 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 08 02:33:16.016322 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 08 02:33:16.016322 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 08 02:33:16.016322 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 08 02:33:16.016322 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Aug 08 02:33:16.016322 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi item = cinderclient( Aug 08 02:33:16.016322 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Aug 08 02:33:16.016322 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Aug 08 02:33:16.016322 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 08 02:33:16.016322 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 08 02:33:16.016322 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 08 02:33:16.016322 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 08 02:33:16.017556 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 08 02:33:16.017556 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 08 02:33:16.017556 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 08 02:33:16.017556 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 08 02:33:16.017556 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume b8281c0f-023d-4904-9fe8-34a52f3a16b4 could not be found. Aug 08 02:33:16.017556 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:33:16.017556 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:33:16.017556 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:33:16.017556 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:33:16.017556 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:33:16.017556 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:33:16.017556 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:33:16.017556 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:33:16.017556 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 08 02:33:16.017556 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 08 02:33:16.017556 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:33:16.017556 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:33:16.017556 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Aug 08 02:33:16.017556 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:33:16.018719 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume b8281c0f-023d-4904-9fe8-34a52f3a16b4 could not be found. Aug 08 02:33:16.018719 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:33:16.018719 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:33:16.018719 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:33:16.018719 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:33:16.018719 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:33:16.018719 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:33:16.018719 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:33:16.018719 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:33:16.018719 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume b8281c0f-023d-4904-9fe8-34a52f3a16b4 could not be found. Aug 08 02:33:16.018719 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:33:16.018719 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-cb01702f-54d1-49e6-8f63-bfeabe92aba5 tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] Returning 404 to user: Volume b8281c0f-023d-4904-9fe8-34a52f3a16b4 could not be found. {{(pid=118351) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:33:16.018719 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-cb01702f-54d1-49e6-8f63-bfeabe92aba5 tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/b8281c0f-023d-4904-9fe8-34a52f3a16b4" status: 404 len: 109 microversion: 2.1 time: 0.036027 Aug 08 02:33:16.018719 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 482/974] 10.222.0.27 () {58 vars in 1245 bytes} [Mon Aug 8 02:33:15 2022] GET /compute/v2.1/os-volumes/b8281c0f-023d-4904-9fe8-34a52f3a16b4 => generated 109 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 08 02:33:16.028649 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-bce8aba1-c0c1-4f5a-a3e6-28ceb04adc64 tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] Calling method '.version_select at 0x7efc74090b80>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:33:16.034919 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell0:SELECT=2 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:16.036142 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell0:SELECT=8 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:16.058712 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.wsgi [None req-bce8aba1-c0c1-4f5a-a3e6-28ceb04adc64 tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] HTTP exception thrown: Volume b8281c0f-023d-4904-9fe8-34a52f3a16b4 could not be found. Aug 08 02:33:16.060831 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-bce8aba1-c0c1-4f5a-a3e6-28ceb04adc64 tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] Request method failure captured: Aug 08 02:33:16.060831 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: request: DELETE /compute/v2.1/os-volumes/b8281c0f-023d-4904-9fe8-34a52f3a16b4 HTTP/1.1 Aug 08 02:33:16.060831 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept: application/json Aug 08 02:33:16.060831 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept-Encoding: identity Aug 08 02:33:16.060831 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Connection: close Aug 08 02:33:16.060831 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Length: 0 Aug 08 02:33:16.060831 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Type: application/json Aug 08 02:33:16.060831 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Host: 10.222.0.27 Aug 08 02:33:16.060831 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Openstack-System-Scope: None Aug 08 02:33:16.060831 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: User-Agent: python-urllib3/1.26.11 Aug 08 02:33:16.060831 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Auth-Token: gAAAAABi8HXhaYxhkLc7eS6ItrppO8DKmi6Y6b2baYkXD4bDhyb85iXbV0QGMMqvoKAed38RxJZ0AdkiY6o33MKBWpg8Cg3O36N5LMUYksSfDFMDpM6ec01Puhd8OfLAZKhPw9P6ReR4M0g9SEDUhnfcdxHwNlgjpBHhYIyZgjyrAbj62eBht8w Aug 08 02:33:16.060831 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Id: None Aug 08 02:33:16.060831 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Name: None Aug 08 02:33:16.060831 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Identity-Status: Confirmed Aug 08 02:33:16.060831 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Is-Admin-Project: True Aug 08 02:33:16.060831 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Id: default Aug 08 02:33:16.060831 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Name: Default Aug 08 02:33:16.060831 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Id: 962280636320471080abda11086f1e9e Aug 08 02:33:16.060831 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-25755299 Aug 08 02:33:16.060831 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Role: reader,member Aug 08 02:33:16.060831 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Roles: reader,member Aug 08 02:33:16.063216 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/962280636320471080abda11086f1e9e"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/962280636320471080abda11086f1e9e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_962280636320471080abda11086f1e9e", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/962280636320471080abda11086f1e9e"}]}] Aug 08 02:33:16.063216 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant: tempest-VolumesSnapshotsTestJSON-25755299 Aug 08 02:33:16.063216 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Id: 962280636320471080abda11086f1e9e Aug 08 02:33:16.063216 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-25755299 Aug 08 02:33:16.063216 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User: tempest-VolumesSnapshotsTestJSON-25755299-project Aug 08 02:33:16.063216 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Id: default Aug 08 02:33:16.063216 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Name: Default Aug 08 02:33:16.063216 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Id: 78e6c594dd7f4cd8b20c1361c1eadcc9 Aug 08 02:33:16.063216 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Name: tempest-VolumesSnapshotsTestJSON-25755299-project Aug 08 02:33:16.063216 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: method: Aug 08 02:33:16.063216 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: exception: Volume b8281c0f-023d-4904-9fe8-34a52f3a16b4 could not be found. Aug 08 02:33:16.063216 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: action_args: {'id': 'b8281c0f-023d-4904-9fe8-34a52f3a16b4'} Aug 08 02:33:16.063216 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:33:16.064343 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:33:16.064343 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 08 02:33:16.064343 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 08 02:33:16.064343 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Aug 08 02:33:16.064343 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Aug 08 02:33:16.064343 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Aug 08 02:33:16.064343 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return self._delete(loc) Aug 08 02:33:16.064343 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Aug 08 02:33:16.064343 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Aug 08 02:33:16.064343 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Aug 08 02:33:16.064343 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Aug 08 02:33:16.064343 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 08 02:33:16.064343 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 08 02:33:16.064343 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 08 02:33:16.064343 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 08 02:33:16.064343 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume b8281c0f-023d-4904-9fe8-34a52f3a16b4 could not be found. (HTTP 404) (Request-ID: req-5f161e47-5f83-4ed7-a692-09acbc451c30) Aug 08 02:33:16.064343 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:33:16.064343 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:33:16.064343 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:33:16.065817 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:33:16.065817 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Aug 08 02:33:16.065817 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Aug 08 02:33:16.065817 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 08 02:33:16.065817 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 08 02:33:16.065817 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Aug 08 02:33:16.065817 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Aug 08 02:33:16.065817 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 08 02:33:16.065817 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 08 02:33:16.065817 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 08 02:33:16.065817 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 08 02:33:16.065817 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Aug 08 02:33:16.065817 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Aug 08 02:33:16.065817 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Aug 08 02:33:16.065817 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return self._delete(loc) Aug 08 02:33:16.065817 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Aug 08 02:33:16.065817 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Aug 08 02:33:16.065817 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Aug 08 02:33:16.065817 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Aug 08 02:33:16.067136 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 08 02:33:16.067136 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 08 02:33:16.067136 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 08 02:33:16.067136 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 08 02:33:16.067136 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume b8281c0f-023d-4904-9fe8-34a52f3a16b4 could not be found. Aug 08 02:33:16.067136 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:33:16.067136 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:33:16.067136 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:33:16.067136 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:33:16.067136 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:33:16.067136 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:33:16.067136 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:33:16.067136 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:33:16.067136 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 08 02:33:16.067136 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 08 02:33:16.067136 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:33:16.067136 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:33:16.067136 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Aug 08 02:33:16.067136 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:33:16.068610 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume b8281c0f-023d-4904-9fe8-34a52f3a16b4 could not be found. Aug 08 02:33:16.068610 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:33:16.068610 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:33:16.068610 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:33:16.068610 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:33:16.068610 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:33:16.068610 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:33:16.068610 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:33:16.068610 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:33:16.068610 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume b8281c0f-023d-4904-9fe8-34a52f3a16b4 could not be found. Aug 08 02:33:16.068610 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:33:16.068610 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-bce8aba1-c0c1-4f5a-a3e6-28ceb04adc64 tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] Returning 404 to user: Volume b8281c0f-023d-4904-9fe8-34a52f3a16b4 could not be found. {{(pid=118352) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:33:16.068610 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-bce8aba1-c0c1-4f5a-a3e6-28ceb04adc64 tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] 10.222.0.27 "DELETE /compute/v2.1/os-volumes/b8281c0f-023d-4904-9fe8-34a52f3a16b4" status: 404 len: 109 microversion: 2.1 time: 0.038550 Aug 08 02:33:16.068610 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 493/975] 10.222.0.27 () {58 vars in 1248 bytes} [Mon Aug 8 02:33:16 2022] DELETE /compute/v2.1/os-volumes/b8281c0f-023d-4904-9fe8-34a52f3a16b4 => generated 109 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 08 02:33:16.073605 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-79cb8265-3e3f-4d1f-954a-5226e52701be tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] Calling method '.version_select at 0x7efc6f7d10d0>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:33:16.101639 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.wsgi [None req-79cb8265-3e3f-4d1f-954a-5226e52701be tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] HTTP exception thrown: Volume b8281c0f-023d-4904-9fe8-34a52f3a16b4 could not be found. Aug 08 02:33:16.103591 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-79cb8265-3e3f-4d1f-954a-5226e52701be tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] Request method failure captured: Aug 08 02:33:16.103591 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: request: GET /compute/v2.1/os-volumes/b8281c0f-023d-4904-9fe8-34a52f3a16b4 HTTP/1.1 Aug 08 02:33:16.103591 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Accept: application/json Aug 08 02:33:16.103591 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Accept-Encoding: identity Aug 08 02:33:16.103591 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Connection: close Aug 08 02:33:16.103591 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Content-Length: 0 Aug 08 02:33:16.103591 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Content-Type: application/json Aug 08 02:33:16.103591 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Host: 10.222.0.27 Aug 08 02:33:16.103591 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Openstack-System-Scope: None Aug 08 02:33:16.103591 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: User-Agent: python-urllib3/1.26.11 Aug 08 02:33:16.103591 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Auth-Token: gAAAAABi8HXhaYxhkLc7eS6ItrppO8DKmi6Y6b2baYkXD4bDhyb85iXbV0QGMMqvoKAed38RxJZ0AdkiY6o33MKBWpg8Cg3O36N5LMUYksSfDFMDpM6ec01Puhd8OfLAZKhPw9P6ReR4M0g9SEDUhnfcdxHwNlgjpBHhYIyZgjyrAbj62eBht8w Aug 08 02:33:16.103591 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Domain-Id: None Aug 08 02:33:16.103591 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Domain-Name: None Aug 08 02:33:16.103591 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Identity-Status: Confirmed Aug 08 02:33:16.103591 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Is-Admin-Project: True Aug 08 02:33:16.103591 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Domain-Id: default Aug 08 02:33:16.103591 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Domain-Name: Default Aug 08 02:33:16.103591 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Id: 962280636320471080abda11086f1e9e Aug 08 02:33:16.103591 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-25755299 Aug 08 02:33:16.103591 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Role: reader,member Aug 08 02:33:16.103591 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Roles: reader,member Aug 08 02:33:16.105253 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/962280636320471080abda11086f1e9e"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/962280636320471080abda11086f1e9e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_962280636320471080abda11086f1e9e", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/962280636320471080abda11086f1e9e"}]}] Aug 08 02:33:16.105253 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant: tempest-VolumesSnapshotsTestJSON-25755299 Aug 08 02:33:16.105253 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant-Id: 962280636320471080abda11086f1e9e Aug 08 02:33:16.105253 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-25755299 Aug 08 02:33:16.105253 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User: tempest-VolumesSnapshotsTestJSON-25755299-project Aug 08 02:33:16.105253 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Domain-Id: default Aug 08 02:33:16.105253 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Domain-Name: Default Aug 08 02:33:16.105253 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Id: 78e6c594dd7f4cd8b20c1361c1eadcc9 Aug 08 02:33:16.105253 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Name: tempest-VolumesSnapshotsTestJSON-25755299-project Aug 08 02:33:16.105253 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: method: Aug 08 02:33:16.105253 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: exception: Volume b8281c0f-023d-4904-9fe8-34a52f3a16b4 could not be found. Aug 08 02:33:16.105253 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: action_args: {'id': 'b8281c0f-023d-4904-9fe8-34a52f3a16b4'} Aug 08 02:33:16.105253 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:33:16.106388 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:33:16.106388 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 08 02:33:16.106388 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 08 02:33:16.106388 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Aug 08 02:33:16.106388 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi item = cinderclient( Aug 08 02:33:16.106388 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Aug 08 02:33:16.106388 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Aug 08 02:33:16.106388 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 08 02:33:16.106388 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 08 02:33:16.106388 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 08 02:33:16.106388 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 08 02:33:16.106388 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 08 02:33:16.106388 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 08 02:33:16.106388 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 08 02:33:16.106388 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 08 02:33:16.106388 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume b8281c0f-023d-4904-9fe8-34a52f3a16b4 could not be found. (HTTP 404) (Request-ID: req-6e04137e-ecfd-4736-b70d-e50d02a00fcc) Aug 08 02:33:16.106388 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:33:16.106388 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:33:16.106388 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:33:16.107806 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:33:16.107806 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Aug 08 02:33:16.107806 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Aug 08 02:33:16.107806 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 08 02:33:16.107806 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 08 02:33:16.107806 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Aug 08 02:33:16.107806 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Aug 08 02:33:16.107806 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 08 02:33:16.107806 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 08 02:33:16.107806 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 08 02:33:16.107806 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 08 02:33:16.107806 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Aug 08 02:33:16.107806 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi item = cinderclient( Aug 08 02:33:16.107806 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Aug 08 02:33:16.107806 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Aug 08 02:33:16.107806 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 08 02:33:16.107806 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 08 02:33:16.107806 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 08 02:33:16.107806 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 08 02:33:16.109386 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 08 02:33:16.109386 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 08 02:33:16.109386 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 08 02:33:16.109386 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 08 02:33:16.109386 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume b8281c0f-023d-4904-9fe8-34a52f3a16b4 could not be found. Aug 08 02:33:16.109386 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:33:16.109386 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:33:16.109386 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:33:16.109386 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:33:16.109386 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:33:16.109386 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:33:16.109386 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:33:16.109386 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:33:16.109386 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 08 02:33:16.109386 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 08 02:33:16.109386 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:33:16.109386 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:33:16.109386 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Aug 08 02:33:16.109386 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:33:16.110746 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume b8281c0f-023d-4904-9fe8-34a52f3a16b4 could not be found. Aug 08 02:33:16.110746 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:33:16.110746 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:33:16.110746 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:33:16.110746 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:33:16.110746 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:33:16.110746 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:33:16.110746 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:33:16.110746 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:33:16.110746 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume b8281c0f-023d-4904-9fe8-34a52f3a16b4 could not be found. Aug 08 02:33:16.110746 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:33:16.110746 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-79cb8265-3e3f-4d1f-954a-5226e52701be tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] Returning 404 to user: Volume b8281c0f-023d-4904-9fe8-34a52f3a16b4 could not be found. {{(pid=118351) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:33:16.110746 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-79cb8265-3e3f-4d1f-954a-5226e52701be tempest-VolumesSnapshotsTestJSON-25755299 tempest-VolumesSnapshotsTestJSON-25755299-project] 10.222.0.27 "GET /compute/v2.1/os-volumes/b8281c0f-023d-4904-9fe8-34a52f3a16b4" status: 404 len: 109 microversion: 2.1 time: 0.036442 Aug 08 02:33:16.110746 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 483/976] 10.222.0.27 () {58 vars in 1245 bytes} [Mon Aug 8 02:33:16 2022] GET /compute/v2.1/os-volumes/b8281c0f-023d-4904-9fe8-34a52f3a16b4 => generated 109 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 08 02:33:16.174970 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-33f71d96-e91b-4d59-960f-4e0446463630 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:33:16.175929 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-33f71d96-e91b-4d59-960f-4e0446463630 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] [instance: d8b00165-4068-46b3-904a-20ca45ed8882] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:33:16.185396 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-33f71d96-e91b-4d59-960f-4e0446463630 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:16.185893 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-33f71d96-e91b-4d59-960f-4e0446463630 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:16.187266 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-33f71d96-e91b-4d59-960f-4e0446463630 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:16.188181 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-33f71d96-e91b-4d59-960f-4e0446463630 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:16.188966 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-33f71d96-e91b-4d59-960f-4e0446463630 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:16.189615 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-33f71d96-e91b-4d59-960f-4e0446463630 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:16.327717 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-33f71d96-e91b-4d59-960f-4e0446463630 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ffcf7b8ddd454d9fa3b32f86906cedd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '414b6a313558425cbd3ed5c6da5405a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:33:16.335198 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-33f71d96-e91b-4d59-960f-4e0446463630 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] 10.222.0.27 "GET /compute/v2.1/servers/d8b00165-4068-46b3-904a-20ca45ed8882" status: 200 len: 1313 microversion: 2.1 time: 0.164716 Aug 08 02:33:16.335759 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 494/977] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:33:16 2022] GET /compute/v2.1/servers/d8b00165-4068-46b3-904a-20ca45ed8882 => generated 1313 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:33:17.356521 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-c8d58ca7-b4d9-4cf6-b2b7-28571731e390 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:33:17.361549 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-c8d58ca7-b4d9-4cf6-b2b7-28571731e390 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] [instance: d8b00165-4068-46b3-904a-20ca45ed8882] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:33:17.363823 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c8d58ca7-b4d9-4cf6-b2b7-28571731e390 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:17.364389 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c8d58ca7-b4d9-4cf6-b2b7-28571731e390 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:17.364829 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c8d58ca7-b4d9-4cf6-b2b7-28571731e390 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:17.365530 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c8d58ca7-b4d9-4cf6-b2b7-28571731e390 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:17.365984 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c8d58ca7-b4d9-4cf6-b2b7-28571731e390 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:17.366438 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-c8d58ca7-b4d9-4cf6-b2b7-28571731e390 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:17.499442 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-c8d58ca7-b4d9-4cf6-b2b7-28571731e390 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ffcf7b8ddd454d9fa3b32f86906cedd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '414b6a313558425cbd3ed5c6da5405a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:33:17.507196 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-c8d58ca7-b4d9-4cf6-b2b7-28571731e390 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] 10.222.0.27 "GET /compute/v2.1/servers/d8b00165-4068-46b3-904a-20ca45ed8882" status: 200 len: 1313 microversion: 2.1 time: 0.155854 Aug 08 02:33:17.507752 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 484/978] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:33:17 2022] GET /compute/v2.1/servers/d8b00165-4068-46b3-904a-20ca45ed8882 => generated 1313 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:33:18.528346 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-88ac4240-7dad-4400-8e18-a8e8d166d95e tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:33:18.528987 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-88ac4240-7dad-4400-8e18-a8e8d166d95e tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] [instance: d8b00165-4068-46b3-904a-20ca45ed8882] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:33:18.535339 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-88ac4240-7dad-4400-8e18-a8e8d166d95e tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:18.535827 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-88ac4240-7dad-4400-8e18-a8e8d166d95e tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:18.536483 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-88ac4240-7dad-4400-8e18-a8e8d166d95e tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:18.537218 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-88ac4240-7dad-4400-8e18-a8e8d166d95e tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:18.537676 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-88ac4240-7dad-4400-8e18-a8e8d166d95e tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:18.538098 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-88ac4240-7dad-4400-8e18-a8e8d166d95e tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:18.674210 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-88ac4240-7dad-4400-8e18-a8e8d166d95e tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ffcf7b8ddd454d9fa3b32f86906cedd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '414b6a313558425cbd3ed5c6da5405a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:33:18.682098 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-88ac4240-7dad-4400-8e18-a8e8d166d95e tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] 10.222.0.27 "GET /compute/v2.1/servers/d8b00165-4068-46b3-904a-20ca45ed8882" status: 200 len: 1313 microversion: 2.1 time: 0.157536 Aug 08 02:33:18.682757 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 495/979] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:33:18 2022] GET /compute/v2.1/servers/d8b00165-4068-46b3-904a-20ca45ed8882 => generated 1313 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:33:19.704120 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-7c878929-6a59-4431-81ee-3bfc37a6dbea tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:33:19.704741 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-7c878929-6a59-4431-81ee-3bfc37a6dbea tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] [instance: d8b00165-4068-46b3-904a-20ca45ed8882] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:33:19.711265 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7c878929-6a59-4431-81ee-3bfc37a6dbea tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:19.711760 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7c878929-6a59-4431-81ee-3bfc37a6dbea tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:19.712192 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7c878929-6a59-4431-81ee-3bfc37a6dbea tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:19.712925 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7c878929-6a59-4431-81ee-3bfc37a6dbea tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:19.713370 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7c878929-6a59-4431-81ee-3bfc37a6dbea tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:19.713967 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7c878929-6a59-4431-81ee-3bfc37a6dbea tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:19.892153 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-7c878929-6a59-4431-81ee-3bfc37a6dbea tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ffcf7b8ddd454d9fa3b32f86906cedd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '414b6a313558425cbd3ed5c6da5405a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:33:19.900215 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-7c878929-6a59-4431-81ee-3bfc37a6dbea tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] 10.222.0.27 "GET /compute/v2.1/servers/d8b00165-4068-46b3-904a-20ca45ed8882" status: 200 len: 1313 microversion: 2.1 time: 0.200975 Aug 08 02:33:19.910619 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 485/980] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:33:19 2022] GET /compute/v2.1/servers/d8b00165-4068-46b3-904a-20ca45ed8882 => generated 1313 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:33:20.079337 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [req-a034e3de-6da4-4048-83c9-afaadd39c4ad req-c740a570-f137-49dd-b674-90f1cd3d6fc8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d8b00165-4068-46b3-904a-20ca45ed8882", "name": "network-vif-plugged", "status": "completed", "tag": "35865bc3-fb2d-47aa-9dca-c85e9076883c"}]} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:33:20.089032 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_api:SELECT=80,nova_api:UPDATE=2,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:20.096834 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-a034e3de-6da4-4048-83c9-afaadd39c4ad req-c740a570-f137-49dd-b674-90f1cd3d6fc8 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:20.097316 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-a034e3de-6da4-4048-83c9-afaadd39c4ad req-c740a570-f137-49dd-b674-90f1cd3d6fc8 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:20.097743 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-a034e3de-6da4-4048-83c9-afaadd39c4ad req-c740a570-f137-49dd-b674-90f1cd3d6fc8 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:20.110420 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.compute.server_external_events [req-a034e3de-6da4-4048-83c9-afaadd39c4ad req-c740a570-f137-49dd-b674-90f1cd3d6fc8 service nova] Creating event network-vif-plugged:35865bc3-fb2d-47aa-9dca-c85e9076883c for instance d8b00165-4068-46b3-904a-20ca45ed8882 on hv-cinder-74014 Aug 08 02:33:20.115632 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [req-a034e3de-6da4-4048-83c9-afaadd39c4ad req-c740a570-f137-49dd-b674-90f1cd3d6fc8 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043590 Aug 08 02:33:20.116242 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 496/981] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:33:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:33:20.922881 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-36d0db0d-459d-49d8-b363-133fe630db1d tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:33:20.923487 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-36d0db0d-459d-49d8-b363-133fe630db1d tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] [instance: d8b00165-4068-46b3-904a-20ca45ed8882] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:33:20.929960 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-36d0db0d-459d-49d8-b363-133fe630db1d tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:20.930444 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-36d0db0d-459d-49d8-b363-133fe630db1d tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:20.930885 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-36d0db0d-459d-49d8-b363-133fe630db1d tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:20.931579 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-36d0db0d-459d-49d8-b363-133fe630db1d tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:20.932084 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-36d0db0d-459d-49d8-b363-133fe630db1d tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:20.932477 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-36d0db0d-459d-49d8-b363-133fe630db1d tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:21.072244 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-36d0db0d-459d-49d8-b363-133fe630db1d tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ffcf7b8ddd454d9fa3b32f86906cedd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '414b6a313558425cbd3ed5c6da5405a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:33:21.080534 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-36d0db0d-459d-49d8-b363-133fe630db1d tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] 10.222.0.27 "GET /compute/v2.1/servers/d8b00165-4068-46b3-904a-20ca45ed8882" status: 200 len: 1313 microversion: 2.1 time: 0.161819 Aug 08 02:33:21.081138 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 486/982] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:33:20 2022] GET /compute/v2.1/servers/d8b00165-4068-46b3-904a-20ca45ed8882 => generated 1313 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:33:22.104123 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-91c2381a-34ce-4d4a-b933-a4dea4e46165 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:33:22.105449 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-91c2381a-34ce-4d4a-b933-a4dea4e46165 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] [instance: d8b00165-4068-46b3-904a-20ca45ed8882] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:33:22.114264 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-91c2381a-34ce-4d4a-b933-a4dea4e46165 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:22.114264 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-91c2381a-34ce-4d4a-b933-a4dea4e46165 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:22.114264 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-91c2381a-34ce-4d4a-b933-a4dea4e46165 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:22.115244 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-91c2381a-34ce-4d4a-b933-a4dea4e46165 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:22.115244 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-91c2381a-34ce-4d4a-b933-a4dea4e46165 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:22.115516 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-91c2381a-34ce-4d4a-b933-a4dea4e46165 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:22.250024 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-91c2381a-34ce-4d4a-b933-a4dea4e46165 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ffcf7b8ddd454d9fa3b32f86906cedd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '414b6a313558425cbd3ed5c6da5405a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:33:22.257584 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-91c2381a-34ce-4d4a-b933-a4dea4e46165 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] 10.222.0.27 "GET /compute/v2.1/servers/d8b00165-4068-46b3-904a-20ca45ed8882" status: 200 len: 1498 microversion: 2.1 time: 0.159177 Aug 08 02:33:22.258154 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 497/983] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:33:22 2022] GET /compute/v2.1/servers/d8b00165-4068-46b3-904a-20ca45ed8882 => generated 1498 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:33:23.196225 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-33c1a532-1c22-4a08-b55c-c1d353484695 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:33:23.196834 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-33c1a532-1c22-4a08-b55c-c1d353484695 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] [instance: d8b00165-4068-46b3-904a-20ca45ed8882] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:33:23.203797 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-33c1a532-1c22-4a08-b55c-c1d353484695 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:23.204304 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-33c1a532-1c22-4a08-b55c-c1d353484695 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:23.208070 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-33c1a532-1c22-4a08-b55c-c1d353484695 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:23.208070 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-33c1a532-1c22-4a08-b55c-c1d353484695 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:23.208070 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-33c1a532-1c22-4a08-b55c-c1d353484695 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:23.208070 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-33c1a532-1c22-4a08-b55c-c1d353484695 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:23.235057 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-33c1a532-1c22-4a08-b55c-c1d353484695 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] [instance: d8b00165-4068-46b3-904a-20ca45ed8882] Going to try to terminate instance {{(pid=118351) delete /opt/stack/nova/nova/compute/api.py:2666}} Aug 08 02:33:23.529225 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-33c1a532-1c22-4a08-b55c-c1d353484695 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] 10.222.0.27 "DELETE /compute/v2.1/servers/d8b00165-4068-46b3-904a-20ca45ed8882" status: 204 len: 0 microversion: 2.1 time: 0.336561 Aug 08 02:33:23.529805 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 487/984] 10.222.0.27 () {58 vars in 1239 bytes} [Mon Aug 8 02:33:23 2022] DELETE /compute/v2.1/servers/d8b00165-4068-46b3-904a-20ca45ed8882 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 08 02:33:23.539226 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-310ed684-e9d3-4eac-9621-be4b3a6996b2 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:33:23.541974 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-310ed684-e9d3-4eac-9621-be4b3a6996b2 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] [instance: d8b00165-4068-46b3-904a-20ca45ed8882] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:33:23.547308 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-310ed684-e9d3-4eac-9621-be4b3a6996b2 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:23.547764 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-310ed684-e9d3-4eac-9621-be4b3a6996b2 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:23.548264 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-310ed684-e9d3-4eac-9621-be4b3a6996b2 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:23.549117 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-310ed684-e9d3-4eac-9621-be4b3a6996b2 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:23.549569 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-310ed684-e9d3-4eac-9621-be4b3a6996b2 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:23.549994 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-310ed684-e9d3-4eac-9621-be4b3a6996b2 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:23.685880 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-310ed684-e9d3-4eac-9621-be4b3a6996b2 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ffcf7b8ddd454d9fa3b32f86906cedd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '414b6a313558425cbd3ed5c6da5405a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:33:23.693819 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-310ed684-e9d3-4eac-9621-be4b3a6996b2 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] 10.222.0.27 "GET /compute/v2.1/servers/d8b00165-4068-46b3-904a-20ca45ed8882" status: 200 len: 1504 microversion: 2.1 time: 0.158834 Aug 08 02:33:23.694376 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 498/985] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:33:23 2022] GET /compute/v2.1/servers/d8b00165-4068-46b3-904a-20ca45ed8882 => generated 1504 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:33:24.715785 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-68af2949-6787-45be-995c-c866c3f03e04 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:33:24.716651 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-68af2949-6787-45be-995c-c866c3f03e04 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] [instance: d8b00165-4068-46b3-904a-20ca45ed8882] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:33:24.723711 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-68af2949-6787-45be-995c-c866c3f03e04 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:24.724230 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-68af2949-6787-45be-995c-c866c3f03e04 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:24.725848 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-68af2949-6787-45be-995c-c866c3f03e04 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:24.725848 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-68af2949-6787-45be-995c-c866c3f03e04 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:24.729834 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-68af2949-6787-45be-995c-c866c3f03e04 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:24.729834 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-68af2949-6787-45be-995c-c866c3f03e04 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:24.871852 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-68af2949-6787-45be-995c-c866c3f03e04 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ffcf7b8ddd454d9fa3b32f86906cedd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '414b6a313558425cbd3ed5c6da5405a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:33:24.881272 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-68af2949-6787-45be-995c-c866c3f03e04 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] 10.222.0.27 "GET /compute/v2.1/servers/d8b00165-4068-46b3-904a-20ca45ed8882" status: 200 len: 1504 microversion: 2.1 time: 0.169609 Aug 08 02:33:24.881845 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 488/986] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:33:24 2022] GET /compute/v2.1/servers/d8b00165-4068-46b3-904a-20ca45ed8882 => generated 1504 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:33:25.534924 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [req-9bf97de6-1029-4d77-ad34-00cef50da29d req-3cfba0ee-76e5-4781-a61a-18dbb7c6e826 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d8b00165-4068-46b3-904a-20ca45ed8882", "name": "network-vif-deleted", "tag": "35865bc3-fb2d-47aa-9dca-c85e9076883c"}]} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:33:25.545880 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-9bf97de6-1029-4d77-ad34-00cef50da29d req-3cfba0ee-76e5-4781-a61a-18dbb7c6e826 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:25.546471 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-9bf97de6-1029-4d77-ad34-00cef50da29d req-3cfba0ee-76e5-4781-a61a-18dbb7c6e826 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:25.547713 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-9bf97de6-1029-4d77-ad34-00cef50da29d req-3cfba0ee-76e5-4781-a61a-18dbb7c6e826 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:25.559105 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.compute.server_external_events [req-9bf97de6-1029-4d77-ad34-00cef50da29d req-3cfba0ee-76e5-4781-a61a-18dbb7c6e826 service nova] Creating event network-vif-deleted:35865bc3-fb2d-47aa-9dca-c85e9076883c for instance d8b00165-4068-46b3-904a-20ca45ed8882 on hv-cinder-74014 Aug 08 02:33:25.564751 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [req-9bf97de6-1029-4d77-ad34-00cef50da29d req-3cfba0ee-76e5-4781-a61a-18dbb7c6e826 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034003 Aug 08 02:33:25.565425 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 499/987] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:33:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:33:25.904127 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-81076cab-c5da-4b9c-a30b-448e8220bf4e tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:33:25.906699 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-81076cab-c5da-4b9c-a30b-448e8220bf4e tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] [instance: d8b00165-4068-46b3-904a-20ca45ed8882] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:33:25.914992 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-81076cab-c5da-4b9c-a30b-448e8220bf4e tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:25.916438 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-81076cab-c5da-4b9c-a30b-448e8220bf4e tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:25.919667 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-81076cab-c5da-4b9c-a30b-448e8220bf4e tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:25.924355 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-81076cab-c5da-4b9c-a30b-448e8220bf4e tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:25.924355 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-81076cab-c5da-4b9c-a30b-448e8220bf4e tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:25.924355 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-81076cab-c5da-4b9c-a30b-448e8220bf4e tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:25.955283 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-81076cab-c5da-4b9c-a30b-448e8220bf4e tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:25.955761 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-81076cab-c5da-4b9c-a30b-448e8220bf4e tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:25.956191 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-81076cab-c5da-4b9c-a30b-448e8220bf4e tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:25.956631 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.objects.instance [None req-81076cab-c5da-4b9c-a30b-448e8220bf4e tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lazy-loading 'fault' on Instance uuid d8b00165-4068-46b3-904a-20ca45ed8882 {{(pid=118351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 08 02:33:25.997862 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-81076cab-c5da-4b9c-a30b-448e8220bf4e tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ffcf7b8ddd454d9fa3b32f86906cedd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '414b6a313558425cbd3ed5c6da5405a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:33:26.006227 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-81076cab-c5da-4b9c-a30b-448e8220bf4e tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] 10.222.0.27 "GET /compute/v2.1/servers/d8b00165-4068-46b3-904a-20ca45ed8882" status: 200 len: 1303 microversion: 2.1 time: 0.106549 Aug 08 02:33:26.006800 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 489/988] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:33:25 2022] GET /compute/v2.1/servers/d8b00165-4068-46b3-904a-20ca45ed8882 => generated 1303 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:33:27.030885 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-9afd1e70-4f0f-497d-a810-a7958d482a53 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:33:27.031770 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-9afd1e70-4f0f-497d-a810-a7958d482a53 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] [instance: d8b00165-4068-46b3-904a-20ca45ed8882] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:33:27.039387 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9afd1e70-4f0f-497d-a810-a7958d482a53 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:27.042319 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9afd1e70-4f0f-497d-a810-a7958d482a53 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:27.042319 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9afd1e70-4f0f-497d-a810-a7958d482a53 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:27.042319 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9afd1e70-4f0f-497d-a810-a7958d482a53 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:27.042319 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9afd1e70-4f0f-497d-a810-a7958d482a53 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:27.042801 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9afd1e70-4f0f-497d-a810-a7958d482a53 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:27.056878 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.wsgi [None req-9afd1e70-4f0f-497d-a810-a7958d482a53 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] HTTP exception thrown: Instance d8b00165-4068-46b3-904a-20ca45ed8882 could not be found. Aug 08 02:33:27.058509 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-9afd1e70-4f0f-497d-a810-a7958d482a53 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Request method failure captured: Aug 08 02:33:27.058509 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: request: GET /compute/v2.1/servers/d8b00165-4068-46b3-904a-20ca45ed8882 HTTP/1.1 Aug 08 02:33:27.058509 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept: application/json Aug 08 02:33:27.058509 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept-Encoding: identity Aug 08 02:33:27.058509 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Connection: close Aug 08 02:33:27.058509 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Length: 0 Aug 08 02:33:27.058509 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Type: application/json Aug 08 02:33:27.058509 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Host: 10.222.0.27 Aug 08 02:33:27.058509 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Openstack-System-Scope: None Aug 08 02:33:27.058509 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: User-Agent: python-urllib3/1.26.11 Aug 08 02:33:27.058509 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Auth-Token: gAAAAABi8HXKG9SYsmkm0nFiOTijFlHCIRv3KiimkAJPRCn3zSSjTRVU4YMV7t3L8udBlfiR-QVKW-AfUNzld8CiOIZpvF3noYv9Le5N6YNq-PtySVD_6RhIYHhQaxBgQBLl96TeJ2Hgc_sJJrh2dbvcaET5TK2rG4EkHkmLZgUdbSwuWPFeoz8 Aug 08 02:33:27.058509 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Id: None Aug 08 02:33:27.058509 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Name: None Aug 08 02:33:27.058509 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Identity-Status: Confirmed Aug 08 02:33:27.058509 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Is-Admin-Project: True Aug 08 02:33:27.058509 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Id: default Aug 08 02:33:27.058509 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Name: Default Aug 08 02:33:27.058509 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Id: 414b6a313558425cbd3ed5c6da5405a9 Aug 08 02:33:27.058509 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Name: tempest-VolumesActionsTest-1042062957 Aug 08 02:33:27.058509 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Role: reader,member Aug 08 02:33:27.058509 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Roles: reader,member Aug 08 02:33:27.060184 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/414b6a313558425cbd3ed5c6da5405a9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/414b6a313558425cbd3ed5c6da5405a9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_414b6a313558425cbd3ed5c6da5405a9", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/414b6a313558425cbd3ed5c6da5405a9"}]}] Aug 08 02:33:27.060184 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant: tempest-VolumesActionsTest-1042062957 Aug 08 02:33:27.060184 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Id: 414b6a313558425cbd3ed5c6da5405a9 Aug 08 02:33:27.060184 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Name: tempest-VolumesActionsTest-1042062957 Aug 08 02:33:27.060184 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User: tempest-VolumesActionsTest-1042062957-project Aug 08 02:33:27.060184 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Id: default Aug 08 02:33:27.060184 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Name: Default Aug 08 02:33:27.060184 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Id: ffcf7b8ddd454d9fa3b32f86906cedd5 Aug 08 02:33:27.060184 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Name: tempest-VolumesActionsTest-1042062957-project Aug 08 02:33:27.060184 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: method: > Aug 08 02:33:27.060184 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: exception: Instance d8b00165-4068-46b3-904a-20ca45ed8882 could not be found. Aug 08 02:33:27.060184 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: action_args: {'id': 'd8b00165-4068-46b3-904a-20ca45ed8882'} Aug 08 02:33:27.061061 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:33:27.061061 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:33:27.061061 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 08 02:33:27.061061 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 08 02:33:27.061061 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 08 02:33:27.061061 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 08 02:33:27.061061 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 08 02:33:27.061061 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 08 02:33:27.061061 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 08 02:33:27.061061 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 08 02:33:27.061061 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d8b00165-4068-46b3-904a-20ca45ed8882 could not be found. Aug 08 02:33:27.061061 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:33:27.061061 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:33:27.061061 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:33:27.061061 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:33:27.061061 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:33:27.061061 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:33:27.061061 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:33:27.061061 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:33:27.061061 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:33:27.062302 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:33:27.062302 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 08 02:33:27.062302 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 08 02:33:27.062302 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 08 02:33:27.062302 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 08 02:33:27.062302 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 08 02:33:27.062302 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:33:27.062302 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d8b00165-4068-46b3-904a-20ca45ed8882 could not be found. Aug 08 02:33:27.062302 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:33:27.062302 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:33:27.062302 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:33:27.062302 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:33:27.062302 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:33:27.062302 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:33:27.062302 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:33:27.062302 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:33:27.062302 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d8b00165-4068-46b3-904a-20ca45ed8882 could not be found. Aug 08 02:33:27.062302 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:33:27.063609 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-9afd1e70-4f0f-497d-a810-a7958d482a53 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] Returning 404 to user: Instance d8b00165-4068-46b3-904a-20ca45ed8882 could not be found. {{(pid=118352) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:33:27.063609 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-9afd1e70-4f0f-497d-a810-a7958d482a53 tempest-VolumesActionsTest-1042062957 tempest-VolumesActionsTest-1042062957-project] 10.222.0.27 "GET /compute/v2.1/servers/d8b00165-4068-46b3-904a-20ca45ed8882" status: 404 len: 111 microversion: 2.1 time: 0.034697 Aug 08 02:33:27.063609 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 500/989] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:33:27 2022] GET /compute/v2.1/servers/d8b00165-4068-46b3-904a-20ca45ed8882 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 08 02:33:58.316501 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-1c699b15-84d9-447b-8c03-cf08168a4c65 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesNegativeTest-instance-490641132", "imageRef": "30833f47-5987-41ca-b5ee-ad5ecafbb265", "flavorRef": "452"}} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:33:58.325684 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_api:SELECT=40,nova_api:UPDATE=1 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:58.326531 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell1:SELECT=51,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:58.377205 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.network.neutron [None req-1c699b15-84d9-447b-8c03-cf08168a4c65 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] validate_networks() for None {{(pid=118351) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 08 02:33:58.727622 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.virt.hardware [None req-1c699b15-84d9-447b-8c03-cf08168a4c65 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Flavor limits 0:0:0 {{(pid=118351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 08 02:33:58.728057 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.virt.hardware [None req-1c699b15-84d9-447b-8c03-cf08168a4c65 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Image limits 0:0:0 {{(pid=118351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 08 02:33:58.728531 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.virt.hardware [None req-1c699b15-84d9-447b-8c03-cf08168a4c65 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Flavor pref 0:0:0 {{(pid=118351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 08 02:33:58.728999 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.virt.hardware [None req-1c699b15-84d9-447b-8c03-cf08168a4c65 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Image pref 0:0:0 {{(pid=118351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 08 02:33:58.729413 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.virt.hardware [None req-1c699b15-84d9-447b-8c03-cf08168a4c65 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118351) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 08 02:33:58.739930 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1c699b15-84d9-447b-8c03-cf08168a4c65 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:58.740422 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1c699b15-84d9-447b-8c03-cf08168a4c65 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:58.740890 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1c699b15-84d9-447b-8c03-cf08168a4c65 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:58.741621 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1c699b15-84d9-447b-8c03-cf08168a4c65 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:33:58.742077 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1c699b15-84d9-447b-8c03-cf08168a4c65 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:33:58.742518 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1c699b15-84d9-447b-8c03-cf08168a4c65 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:33:58.762435 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.quota [None req-1c699b15-84d9-447b-8c03-cf08168a4c65 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Getting quotas for project c30c9b3f01e740c2a329123736eb2e5e. Resources: {'ram', 'cores', 'instances'} {{(pid=118351) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 08 02:33:58.769341 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.quota [None req-1c699b15-84d9-447b-8c03-cf08168a4c65 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Getting quotas for user 0d75fc1643de49c680b5cb6b977fdb0c and project c30c9b3f01e740c2a329123736eb2e5e. Resources: {'ram', 'cores', 'instances'} {{(pid=118351) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 08 02:33:58.787828 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-1c699b15-84d9-447b-8c03-cf08168a4c65 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Going to run 1 instances... {{(pid=118351) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Aug 08 02:33:58.790218 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-1c699b15-84d9-447b-8c03-cf08168a4c65 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] [instance: d1c59bd2-0519-4579-85f3-532b1b8400a8] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='30833f47-5987-41ca-b5ee-ad5ecafbb265',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118351) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Aug 08 02:33:58.909064 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-1c699b15-84d9-447b-8c03-cf08168a4c65 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.596499 Aug 08 02:33:58.911473 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 490/990] 10.222.0.27 () {60 vars in 1147 bytes} [Mon Aug 8 02:33:58 2022] POST /compute/v2.1/servers => generated 380 bytes in 599 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Aug 08 02:33:58.923112 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-c4084471-8d3c-42c9-93f8-fa75b886279b tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:33:58.925363 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-c4084471-8d3c-42c9-93f8-fa75b886279b tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] [instance: d1c59bd2-0519-4579-85f3-532b1b8400a8] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:33:58.933196 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_api:SELECT=9 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:58.933196 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell1:SELECT=41 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:58.995652 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-c4084471-8d3c-42c9-93f8-fa75b886279b tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d75fc1643de49c680b5cb6b977fdb0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c30c9b3f01e740c2a329123736eb2e5e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:33:59.006495 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-c4084471-8d3c-42c9-93f8-fa75b886279b tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] 10.222.0.27 "GET /compute/v2.1/servers/d1c59bd2-0519-4579-85f3-532b1b8400a8" status: 200 len: 1214 microversion: 2.1 time: 0.088297 Aug 08 02:33:59.024902 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 501/991] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:33:58 2022] GET /compute/v2.1/servers/d1c59bd2-0519-4579-85f3-532b1b8400a8 => generated 1214 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:00.031344 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-ae6cec79-db55-4c0b-bab9-d3dacd7f3677 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:00.031344 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-ae6cec79-db55-4c0b-bab9-d3dacd7f3677 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] [instance: d1c59bd2-0519-4579-85f3-532b1b8400a8] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:00.127797 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-ae6cec79-db55-4c0b-bab9-d3dacd7f3677 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d75fc1643de49c680b5cb6b977fdb0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c30c9b3f01e740c2a329123736eb2e5e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:00.137437 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-ae6cec79-db55-4c0b-bab9-d3dacd7f3677 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] 10.222.0.27 "GET /compute/v2.1/servers/d1c59bd2-0519-4579-85f3-532b1b8400a8" status: 200 len: 1214 microversion: 2.1 time: 0.111752 Aug 08 02:34:00.142939 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 491/992] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:00 2022] GET /compute/v2.1/servers/d1c59bd2-0519-4579-85f3-532b1b8400a8 => generated 1214 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:01.159384 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-db122b5c-02ab-4a9c-a988-bc313552e978 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:01.161016 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-db122b5c-02ab-4a9c-a988-bc313552e978 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] [instance: d1c59bd2-0519-4579-85f3-532b1b8400a8] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:01.211750 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-db122b5c-02ab-4a9c-a988-bc313552e978 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:01.212700 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-db122b5c-02ab-4a9c-a988-bc313552e978 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:01.213660 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-db122b5c-02ab-4a9c-a988-bc313552e978 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:01.214643 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-db122b5c-02ab-4a9c-a988-bc313552e978 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:01.215329 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-db122b5c-02ab-4a9c-a988-bc313552e978 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:01.215979 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-db122b5c-02ab-4a9c-a988-bc313552e978 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:01.609716 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-db122b5c-02ab-4a9c-a988-bc313552e978 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d75fc1643de49c680b5cb6b977fdb0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c30c9b3f01e740c2a329123736eb2e5e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:01.621775 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-db122b5c-02ab-4a9c-a988-bc313552e978 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] 10.222.0.27 "GET /compute/v2.1/servers/d1c59bd2-0519-4579-85f3-532b1b8400a8" status: 200 len: 1266 microversion: 2.1 time: 0.467315 Aug 08 02:34:01.622572 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 502/993] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:01 2022] GET /compute/v2.1/servers/d1c59bd2-0519-4579-85f3-532b1b8400a8 => generated 1266 bytes in 469 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:02.647373 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-f1acf3b5-e209-4975-b499-290465d98285 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:02.648585 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-f1acf3b5-e209-4975-b499-290465d98285 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] [instance: d1c59bd2-0519-4579-85f3-532b1b8400a8] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:02.665423 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f1acf3b5-e209-4975-b499-290465d98285 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:02.666313 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f1acf3b5-e209-4975-b499-290465d98285 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:02.666958 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f1acf3b5-e209-4975-b499-290465d98285 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:02.667866 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f1acf3b5-e209-4975-b499-290465d98285 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:02.668509 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f1acf3b5-e209-4975-b499-290465d98285 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:02.669214 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f1acf3b5-e209-4975-b499-290465d98285 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:03.006474 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-f1acf3b5-e209-4975-b499-290465d98285 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d75fc1643de49c680b5cb6b977fdb0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c30c9b3f01e740c2a329123736eb2e5e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:03.017945 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-f1acf3b5-e209-4975-b499-290465d98285 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] 10.222.0.27 "GET /compute/v2.1/servers/d1c59bd2-0519-4579-85f3-532b1b8400a8" status: 200 len: 1272 microversion: 2.1 time: 0.375942 Aug 08 02:34:03.024557 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 492/994] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:02 2022] GET /compute/v2.1/servers/d1c59bd2-0519-4579-85f3-532b1b8400a8 => generated 1272 bytes in 377 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:04.040499 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-fc0cbc79-9617-4513-9c27-903e8993c721 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:04.042200 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-fc0cbc79-9617-4513-9c27-903e8993c721 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] [instance: d1c59bd2-0519-4579-85f3-532b1b8400a8] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:04.064240 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-fc0cbc79-9617-4513-9c27-903e8993c721 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:04.066990 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-fc0cbc79-9617-4513-9c27-903e8993c721 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:04.067752 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-fc0cbc79-9617-4513-9c27-903e8993c721 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:04.068696 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-fc0cbc79-9617-4513-9c27-903e8993c721 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:04.069338 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-fc0cbc79-9617-4513-9c27-903e8993c721 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:04.069931 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-fc0cbc79-9617-4513-9c27-903e8993c721 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:04.181475 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-fc0cbc79-9617-4513-9c27-903e8993c721 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d75fc1643de49c680b5cb6b977fdb0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c30c9b3f01e740c2a329123736eb2e5e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:04.193948 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-fc0cbc79-9617-4513-9c27-903e8993c721 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] 10.222.0.27 "GET /compute/v2.1/servers/d1c59bd2-0519-4579-85f3-532b1b8400a8" status: 200 len: 1272 microversion: 2.1 time: 0.158212 Aug 08 02:34:04.194555 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 503/995] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:04 2022] GET /compute/v2.1/servers/d1c59bd2-0519-4579-85f3-532b1b8400a8 => generated 1272 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:05.216320 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-e123b720-0d10-4a58-92f0-2cdc8d7963bc tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:05.217175 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-e123b720-0d10-4a58-92f0-2cdc8d7963bc tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] [instance: d1c59bd2-0519-4579-85f3-532b1b8400a8] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:05.224875 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e123b720-0d10-4a58-92f0-2cdc8d7963bc tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:05.225374 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e123b720-0d10-4a58-92f0-2cdc8d7963bc tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:05.225816 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e123b720-0d10-4a58-92f0-2cdc8d7963bc tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:05.226525 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e123b720-0d10-4a58-92f0-2cdc8d7963bc tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:05.227120 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e123b720-0d10-4a58-92f0-2cdc8d7963bc tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:05.227554 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e123b720-0d10-4a58-92f0-2cdc8d7963bc tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:05.452839 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-e123b720-0d10-4a58-92f0-2cdc8d7963bc tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d75fc1643de49c680b5cb6b977fdb0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c30c9b3f01e740c2a329123736eb2e5e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:05.467674 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-e123b720-0d10-4a58-92f0-2cdc8d7963bc tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] 10.222.0.27 "GET /compute/v2.1/servers/d1c59bd2-0519-4579-85f3-532b1b8400a8" status: 200 len: 1314 microversion: 2.1 time: 0.256094 Aug 08 02:34:05.468367 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 493/996] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:05 2022] GET /compute/v2.1/servers/d1c59bd2-0519-4579-85f3-532b1b8400a8 => generated 1314 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:06.095333 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [req-c7cb38f4-5882-47c9-bee6-8768f4780463 req-f47710cf-e04b-47e0-9072-84aa5c12b6e6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d1c59bd2-0519-4579-85f3-532b1b8400a8", "tag": "419e2ca2-827d-458a-8c88-d8c3a45cfcae"}]} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:34:06.118429 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-c7cb38f4-5882-47c9-bee6-8768f4780463 req-f47710cf-e04b-47e0-9072-84aa5c12b6e6 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:06.118838 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-c7cb38f4-5882-47c9-bee6-8768f4780463 req-f47710cf-e04b-47e0-9072-84aa5c12b6e6 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:06.119274 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-c7cb38f4-5882-47c9-bee6-8768f4780463 req-f47710cf-e04b-47e0-9072-84aa5c12b6e6 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:06.140149 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.compute.server_external_events [req-c7cb38f4-5882-47c9-bee6-8768f4780463 req-f47710cf-e04b-47e0-9072-84aa5c12b6e6 service nova] Creating event network-changed:419e2ca2-827d-458a-8c88-d8c3a45cfcae for instance d1c59bd2-0519-4579-85f3-532b1b8400a8 on hv-cinder-74014 Aug 08 02:34:06.154610 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [req-c7cb38f4-5882-47c9-bee6-8768f4780463 req-f47710cf-e04b-47e0-9072-84aa5c12b6e6 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.062741 Aug 08 02:34:06.155541 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 504/997] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:34:06 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:34:06.494625 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-237d3eb3-5dd0-424b-b85a-569f7971e285 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:06.496224 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-237d3eb3-5dd0-424b-b85a-569f7971e285 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] [instance: d1c59bd2-0519-4579-85f3-532b1b8400a8] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:06.511180 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-237d3eb3-5dd0-424b-b85a-569f7971e285 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:06.513217 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-237d3eb3-5dd0-424b-b85a-569f7971e285 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:06.513827 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-237d3eb3-5dd0-424b-b85a-569f7971e285 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:06.514698 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-237d3eb3-5dd0-424b-b85a-569f7971e285 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:06.517246 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-237d3eb3-5dd0-424b-b85a-569f7971e285 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:06.517856 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-237d3eb3-5dd0-424b-b85a-569f7971e285 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:06.820716 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-237d3eb3-5dd0-424b-b85a-569f7971e285 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d75fc1643de49c680b5cb6b977fdb0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c30c9b3f01e740c2a329123736eb2e5e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:06.828951 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-237d3eb3-5dd0-424b-b85a-569f7971e285 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] 10.222.0.27 "GET /compute/v2.1/servers/d1c59bd2-0519-4579-85f3-532b1b8400a8" status: 200 len: 1314 microversion: 2.1 time: 0.338809 Aug 08 02:34:06.843886 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 494/998] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:06 2022] GET /compute/v2.1/servers/d1c59bd2-0519-4579-85f3-532b1b8400a8 => generated 1314 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:07.852257 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-fe65503e-9c78-4234-9d82-eff641024a40 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:07.853758 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-fe65503e-9c78-4234-9d82-eff641024a40 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] [instance: d1c59bd2-0519-4579-85f3-532b1b8400a8] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:07.865152 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-fe65503e-9c78-4234-9d82-eff641024a40 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:07.870428 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-fe65503e-9c78-4234-9d82-eff641024a40 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:07.875422 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-fe65503e-9c78-4234-9d82-eff641024a40 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:07.880367 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-fe65503e-9c78-4234-9d82-eff641024a40 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:07.880367 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-fe65503e-9c78-4234-9d82-eff641024a40 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:07.880367 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-fe65503e-9c78-4234-9d82-eff641024a40 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:08.179779 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-fe65503e-9c78-4234-9d82-eff641024a40 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d75fc1643de49c680b5cb6b977fdb0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c30c9b3f01e740c2a329123736eb2e5e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:08.190341 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-fe65503e-9c78-4234-9d82-eff641024a40 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] 10.222.0.27 "GET /compute/v2.1/servers/d1c59bd2-0519-4579-85f3-532b1b8400a8" status: 200 len: 1314 microversion: 2.1 time: 0.342959 Aug 08 02:34:08.190937 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 505/999] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:07 2022] GET /compute/v2.1/servers/d1c59bd2-0519-4579-85f3-532b1b8400a8 => generated 1314 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:08.752327 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-2de3f6b9-1ea5-4c30-9174-ba8696553a75 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:08.752871 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-2de3f6b9-1ea5-4c30-9174-ba8696553a75 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:08.755574 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell0:SELECT=2 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:08.758229 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [req-a732ffe8-e1b0-48e2-ad2b-ce739ae1239d req-076dc8f5-dde4-4fb3-923e-3c392b195d00 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "077e7935-3361-4ff5-9367-ba162efac09c", "tag": "3051cabc-48a3-4055-ae74-0ae74aa73005"}]} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:34:08.775614 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-a732ffe8-e1b0-48e2-ad2b-ce739ae1239d req-076dc8f5-dde4-4fb3-923e-3c392b195d00 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:08.777137 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-a732ffe8-e1b0-48e2-ad2b-ce739ae1239d req-076dc8f5-dde4-4fb3-923e-3c392b195d00 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:08.777598 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-a732ffe8-e1b0-48e2-ad2b-ce739ae1239d req-076dc8f5-dde4-4fb3-923e-3c392b195d00 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:08.798064 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2de3f6b9-1ea5-4c30-9174-ba8696553a75 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:08.798562 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2de3f6b9-1ea5-4c30-9174-ba8696553a75 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:08.798765 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.compute.server_external_events [req-a732ffe8-e1b0-48e2-ad2b-ce739ae1239d req-076dc8f5-dde4-4fb3-923e-3c392b195d00 service nova] Creating event network-changed:3051cabc-48a3-4055-ae74-0ae74aa73005 for instance 077e7935-3361-4ff5-9367-ba162efac09c on hv-cinder-74014 Aug 08 02:34:08.809883 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [req-a732ffe8-e1b0-48e2-ad2b-ce739ae1239d req-076dc8f5-dde4-4fb3-923e-3c392b195d00 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.055231 Aug 08 02:34:08.810055 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2de3f6b9-1ea5-4c30-9174-ba8696553a75 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:08.810851 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2de3f6b9-1ea5-4c30-9174-ba8696553a75 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:08.811543 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2de3f6b9-1ea5-4c30-9174-ba8696553a75 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:08.812969 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 506/1000] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:34:08 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:34:08.814368 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2de3f6b9-1ea5-4c30-9174-ba8696553a75 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:08.878472 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-2de3f6b9-1ea5-4c30-9174-ba8696553a75 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Going to try to terminate instance {{(pid=118351) delete /opt/stack/nova/nova/compute/api.py:2666}} Aug 08 02:34:09.213264 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell0:SELECT=2 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:09.218605 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-21968302-6226-486c-959b-51b7ef89d622 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:09.219647 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-21968302-6226-486c-959b-51b7ef89d622 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] [instance: d1c59bd2-0519-4579-85f3-532b1b8400a8] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:09.229857 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-21968302-6226-486c-959b-51b7ef89d622 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:09.230693 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-21968302-6226-486c-959b-51b7ef89d622 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:09.231356 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-21968302-6226-486c-959b-51b7ef89d622 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:09.232346 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-21968302-6226-486c-959b-51b7ef89d622 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:09.233050 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-21968302-6226-486c-959b-51b7ef89d622 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:09.233704 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-21968302-6226-486c-959b-51b7ef89d622 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:09.255485 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-2de3f6b9-1ea5-4c30-9174-ba8696553a75 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "DELETE /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 204 len: 0 microversion: 2.1 time: 0.506590 Aug 08 02:34:09.256884 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 495/1001] 10.222.0.27 () {58 vars in 1239 bytes} [Mon Aug 8 02:34:08 2022] DELETE /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 0 bytes in 510 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 08 02:34:09.274757 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-d33c8e7e-aae9-4839-8a64-9739d6622126 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:09.276351 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-d33c8e7e-aae9-4839-8a64-9739d6622126 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:09.284056 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d33c8e7e-aae9-4839-8a64-9739d6622126 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:09.285383 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d33c8e7e-aae9-4839-8a64-9739d6622126 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:09.286005 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d33c8e7e-aae9-4839-8a64-9739d6622126 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:09.287006 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d33c8e7e-aae9-4839-8a64-9739d6622126 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:09.287646 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d33c8e7e-aae9-4839-8a64-9739d6622126 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:09.288278 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d33c8e7e-aae9-4839-8a64-9739d6622126 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:09.494707 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-21968302-6226-486c-959b-51b7ef89d622 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d75fc1643de49c680b5cb6b977fdb0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c30c9b3f01e740c2a329123736eb2e5e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:09.507305 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-21968302-6226-486c-959b-51b7ef89d622 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] 10.222.0.27 "GET /compute/v2.1/servers/d1c59bd2-0519-4579-85f3-532b1b8400a8" status: 200 len: 1314 microversion: 2.1 time: 0.295469 Aug 08 02:34:09.507904 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 507/1002] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:09 2022] GET /compute/v2.1/servers/d1c59bd2-0519-4579-85f3-532b1b8400a8 => generated 1314 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:09.518017 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-d33c8e7e-aae9-4839-8a64-9739d6622126 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:09.534110 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-d33c8e7e-aae9-4839-8a64-9739d6622126 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1567 microversion: 2.1 time: 0.271295 Aug 08 02:34:09.557676 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 496/1003] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:09 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1567 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:10.529712 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-8ef7dcf0-ab65-4d65-b505-5415a40c5b71 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:10.530335 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-8ef7dcf0-ab65-4d65-b505-5415a40c5b71 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] [instance: d1c59bd2-0519-4579-85f3-532b1b8400a8] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:10.540642 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8ef7dcf0-ab65-4d65-b505-5415a40c5b71 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:10.541173 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8ef7dcf0-ab65-4d65-b505-5415a40c5b71 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:10.541622 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8ef7dcf0-ab65-4d65-b505-5415a40c5b71 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:10.542336 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8ef7dcf0-ab65-4d65-b505-5415a40c5b71 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:10.542798 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8ef7dcf0-ab65-4d65-b505-5415a40c5b71 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:10.543235 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8ef7dcf0-ab65-4d65-b505-5415a40c5b71 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:10.557491 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell0:SELECT=8 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:10.563261 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-4f04c6c9-aab1-4187-a04c-dc4696d2a2f2 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:10.564155 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-4f04c6c9-aab1-4187-a04c-dc4696d2a2f2 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:10.580239 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4f04c6c9-aab1-4187-a04c-dc4696d2a2f2 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:10.581144 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4f04c6c9-aab1-4187-a04c-dc4696d2a2f2 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:10.582654 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4f04c6c9-aab1-4187-a04c-dc4696d2a2f2 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:10.583627 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4f04c6c9-aab1-4187-a04c-dc4696d2a2f2 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:10.585309 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4f04c6c9-aab1-4187-a04c-dc4696d2a2f2 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:10.594701 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4f04c6c9-aab1-4187-a04c-dc4696d2a2f2 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:11.521545 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-4f04c6c9-aab1-4187-a04c-dc4696d2a2f2 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:11.545283 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-8ef7dcf0-ab65-4d65-b505-5415a40c5b71 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d75fc1643de49c680b5cb6b977fdb0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c30c9b3f01e740c2a329123736eb2e5e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:11.555207 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-4f04c6c9-aab1-4187-a04c-dc4696d2a2f2 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1567 microversion: 2.1 time: 0.984856 Aug 08 02:34:11.558167 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 497/1004] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:10 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1567 bytes in 1002 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:11.559553 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-8ef7dcf0-ab65-4d65-b505-5415a40c5b71 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] 10.222.0.27 "GET /compute/v2.1/servers/d1c59bd2-0519-4579-85f3-532b1b8400a8" status: 200 len: 1314 microversion: 2.1 time: 1.030660 Aug 08 02:34:11.560762 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 508/1005] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:10 2022] GET /compute/v2.1/servers/d1c59bd2-0519-4579-85f3-532b1b8400a8 => generated 1314 bytes in 1037 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:12.585276 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-f9546008-c07c-4201-8eec-c1ba9c8c0374 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:12.588950 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-f9546008-c07c-4201-8eec-c1ba9c8c0374 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] [instance: d1c59bd2-0519-4579-85f3-532b1b8400a8] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:12.589345 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-908714c2-0cd9-40dc-acb5-7a86352e557d tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:12.590639 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-908714c2-0cd9-40dc-acb5-7a86352e557d tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:12.615712 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f9546008-c07c-4201-8eec-c1ba9c8c0374 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:12.628004 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-908714c2-0cd9-40dc-acb5-7a86352e557d tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:12.629992 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-908714c2-0cd9-40dc-acb5-7a86352e557d tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:12.630921 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f9546008-c07c-4201-8eec-c1ba9c8c0374 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.015s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:12.632440 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f9546008-c07c-4201-8eec-c1ba9c8c0374 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:12.634029 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-908714c2-0cd9-40dc-acb5-7a86352e557d tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:12.635840 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f9546008-c07c-4201-8eec-c1ba9c8c0374 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:12.640316 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-908714c2-0cd9-40dc-acb5-7a86352e557d tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:12.641449 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f9546008-c07c-4201-8eec-c1ba9c8c0374 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:12.642179 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-908714c2-0cd9-40dc-acb5-7a86352e557d tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:12.648389 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-908714c2-0cd9-40dc-acb5-7a86352e557d tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:12.649293 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f9546008-c07c-4201-8eec-c1ba9c8c0374 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:12.971851 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-908714c2-0cd9-40dc-acb5-7a86352e557d tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:12.990120 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-908714c2-0cd9-40dc-acb5-7a86352e557d tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1381 microversion: 2.1 time: 0.414332 Aug 08 02:34:12.991049 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 498/1006] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:12 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1381 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:13.000822 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [req-10acfaa7-d37b-47d0-b679-9d0747940e47 req-e4bd254f-c560-45be-afa6-32460791cf8c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "077e7935-3361-4ff5-9367-ba162efac09c", "name": "network-vif-deleted", "tag": "3051cabc-48a3-4055-ae74-0ae74aa73005"}]} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:34:13.013064 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-10acfaa7-d37b-47d0-b679-9d0747940e47 req-e4bd254f-c560-45be-afa6-32460791cf8c service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:13.013450 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-10acfaa7-d37b-47d0-b679-9d0747940e47 req-e4bd254f-c560-45be-afa6-32460791cf8c service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:13.013893 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-10acfaa7-d37b-47d0-b679-9d0747940e47 req-e4bd254f-c560-45be-afa6-32460791cf8c service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:13.036457 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.compute.server_external_events [req-10acfaa7-d37b-47d0-b679-9d0747940e47 req-e4bd254f-c560-45be-afa6-32460791cf8c service nova] Creating event network-vif-deleted:3051cabc-48a3-4055-ae74-0ae74aa73005 for instance 077e7935-3361-4ff5-9367-ba162efac09c on hv-cinder-74014 Aug 08 02:34:13.045504 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [req-10acfaa7-d37b-47d0-b679-9d0747940e47 req-e4bd254f-c560-45be-afa6-32460791cf8c service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.048129 Aug 08 02:34:13.046392 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 499/1007] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:34:12 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:34:13.051991 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-f9546008-c07c-4201-8eec-c1ba9c8c0374 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d75fc1643de49c680b5cb6b977fdb0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c30c9b3f01e740c2a329123736eb2e5e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:13.086498 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-f9546008-c07c-4201-8eec-c1ba9c8c0374 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] 10.222.0.27 "GET /compute/v2.1/servers/d1c59bd2-0519-4579-85f3-532b1b8400a8" status: 200 len: 1314 microversion: 2.1 time: 0.507621 Aug 08 02:34:13.087341 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 509/1008] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:12 2022] GET /compute/v2.1/servers/d1c59bd2-0519-4579-85f3-532b1b8400a8 => generated 1314 bytes in 509 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:13.445430 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [req-3b28e463-e382-4416-b887-33e58fed4c68 req-06cbd57a-6d94-42f4-add5-25ea4c0209fb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d1c59bd2-0519-4579-85f3-532b1b8400a8", "name": "network-vif-plugged", "status": "completed", "tag": "419e2ca2-827d-458a-8c88-d8c3a45cfcae"}]} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:34:13.465535 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-3b28e463-e382-4416-b887-33e58fed4c68 req-06cbd57a-6d94-42f4-add5-25ea4c0209fb service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:13.466037 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-3b28e463-e382-4416-b887-33e58fed4c68 req-06cbd57a-6d94-42f4-add5-25ea4c0209fb service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:13.466481 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-3b28e463-e382-4416-b887-33e58fed4c68 req-06cbd57a-6d94-42f4-add5-25ea4c0209fb service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:13.496816 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.compute.server_external_events [req-3b28e463-e382-4416-b887-33e58fed4c68 req-06cbd57a-6d94-42f4-add5-25ea4c0209fb service nova] Creating event network-vif-plugged:419e2ca2-827d-458a-8c88-d8c3a45cfcae for instance d1c59bd2-0519-4579-85f3-532b1b8400a8 on hv-cinder-74014 Aug 08 02:34:13.517406 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [req-3b28e463-e382-4416-b887-33e58fed4c68 req-06cbd57a-6d94-42f4-add5-25ea4c0209fb service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.070681 Aug 08 02:34:13.526638 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 500/1009] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:34:13 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:34:14.011483 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-2457966d-4e1b-46c8-90bf-1d78ed727d70 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:14.013075 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-2457966d-4e1b-46c8-90bf-1d78ed727d70 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:14.022786 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2457966d-4e1b-46c8-90bf-1d78ed727d70 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:14.023628 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2457966d-4e1b-46c8-90bf-1d78ed727d70 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:14.024404 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2457966d-4e1b-46c8-90bf-1d78ed727d70 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:14.025388 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2457966d-4e1b-46c8-90bf-1d78ed727d70 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:14.026892 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2457966d-4e1b-46c8-90bf-1d78ed727d70 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:14.027629 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2457966d-4e1b-46c8-90bf-1d78ed727d70 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:14.114915 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-e5cc415c-a9c2-4747-8409-a9b76e7cd48e tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:14.115942 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-e5cc415c-a9c2-4747-8409-a9b76e7cd48e tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] [instance: d1c59bd2-0519-4579-85f3-532b1b8400a8] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:14.129007 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e5cc415c-a9c2-4747-8409-a9b76e7cd48e tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:14.139112 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e5cc415c-a9c2-4747-8409-a9b76e7cd48e tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:14.141213 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e5cc415c-a9c2-4747-8409-a9b76e7cd48e tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:14.147936 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e5cc415c-a9c2-4747-8409-a9b76e7cd48e tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:14.150595 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e5cc415c-a9c2-4747-8409-a9b76e7cd48e tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:14.152104 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e5cc415c-a9c2-4747-8409-a9b76e7cd48e tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:14.156723 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2457966d-4e1b-46c8-90bf-1d78ed727d70 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:14.157516 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2457966d-4e1b-46c8-90bf-1d78ed727d70 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:14.158198 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2457966d-4e1b-46c8-90bf-1d78ed727d70 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:14.158924 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.objects.instance [None req-2457966d-4e1b-46c8-90bf-1d78ed727d70 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lazy-loading 'fault' on Instance uuid 077e7935-3361-4ff5-9367-ba162efac09c {{(pid=118352) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 08 02:34:14.221551 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-2457966d-4e1b-46c8-90bf-1d78ed727d70 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8be4cb4f1c594f42825eb40985a29cac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa4edbfca7b749ff9be3268fa6b81264', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:14.232113 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-2457966d-4e1b-46c8-90bf-1d78ed727d70 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 200 len: 1224 microversion: 2.1 time: 0.224783 Aug 08 02:34:14.233460 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 510/1010] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:14 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 1224 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:14.521028 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-e5cc415c-a9c2-4747-8409-a9b76e7cd48e tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d75fc1643de49c680b5cb6b977fdb0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c30c9b3f01e740c2a329123736eb2e5e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:14.535596 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-e5cc415c-a9c2-4747-8409-a9b76e7cd48e tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] 10.222.0.27 "GET /compute/v2.1/servers/d1c59bd2-0519-4579-85f3-532b1b8400a8" status: 200 len: 1314 microversion: 2.1 time: 0.424188 Aug 08 02:34:14.549565 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 501/1011] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:14 2022] GET /compute/v2.1/servers/d1c59bd2-0519-4579-85f3-532b1b8400a8 => generated 1314 bytes in 441 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:34:15.261994 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-a23b07e8-c8d7-4642-b376-bf7cd4b05e96 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:15.262621 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-a23b07e8-c8d7-4642-b376-bf7cd4b05e96 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] [instance: 077e7935-3361-4ff5-9367-ba162efac09c] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:15.280145 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a23b07e8-c8d7-4642-b376-bf7cd4b05e96 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:15.280145 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a23b07e8-c8d7-4642-b376-bf7cd4b05e96 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:15.280145 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a23b07e8-c8d7-4642-b376-bf7cd4b05e96 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:15.281903 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a23b07e8-c8d7-4642-b376-bf7cd4b05e96 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:15.282413 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a23b07e8-c8d7-4642-b376-bf7cd4b05e96 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:15.283108 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-a23b07e8-c8d7-4642-b376-bf7cd4b05e96 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:15.311235 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.wsgi [None req-a23b07e8-c8d7-4642-b376-bf7cd4b05e96 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] HTTP exception thrown: Instance 077e7935-3361-4ff5-9367-ba162efac09c could not be found. Aug 08 02:34:15.314366 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-a23b07e8-c8d7-4642-b376-bf7cd4b05e96 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Request method failure captured: Aug 08 02:34:15.314366 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: request: GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c HTTP/1.1 Aug 08 02:34:15.314366 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept: application/json Aug 08 02:34:15.314366 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept-Encoding: identity Aug 08 02:34:15.314366 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Connection: close Aug 08 02:34:15.314366 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Length: 0 Aug 08 02:34:15.314366 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Type: application/json Aug 08 02:34:15.314366 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Host: 10.222.0.27 Aug 08 02:34:15.314366 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Openstack-System-Scope: None Aug 08 02:34:15.314366 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: User-Agent: python-urllib3/1.26.11 Aug 08 02:34:15.314366 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Auth-Token: gAAAAABi8HTgkMXxtFqy-IKFcn_8U9lXb2gd5vG3GEOEqZKIGOIFBbQodb97WswXwKNt-6y236booJ1ffRfb33vnayR2EklOmyS1LGOdoDSuSNHvH-MwwR0xyjv5NTc0bLWojY6aCFrB5HtsKnyChigo_1NTpymdS_3j1kaa8zmCNeW48_goLRM Aug 08 02:34:15.314366 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Id: None Aug 08 02:34:15.314366 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Name: None Aug 08 02:34:15.314366 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Identity-Status: Confirmed Aug 08 02:34:15.314366 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Is-Admin-Project: True Aug 08 02:34:15.314366 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Id: default Aug 08 02:34:15.314366 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Name: Default Aug 08 02:34:15.314366 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Id: fa4edbfca7b749ff9be3268fa6b81264 Aug 08 02:34:15.314366 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Name: tempest-TestShelveInstance-2138144306 Aug 08 02:34:15.314366 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Role: reader,member Aug 08 02:34:15.314366 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Roles: reader,member Aug 08 02:34:15.315919 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/fa4edbfca7b749ff9be3268fa6b81264"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/fa4edbfca7b749ff9be3268fa6b81264"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_fa4edbfca7b749ff9be3268fa6b81264", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/fa4edbfca7b749ff9be3268fa6b81264"}]}] Aug 08 02:34:15.315919 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant: tempest-TestShelveInstance-2138144306 Aug 08 02:34:15.315919 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Id: fa4edbfca7b749ff9be3268fa6b81264 Aug 08 02:34:15.315919 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Name: tempest-TestShelveInstance-2138144306 Aug 08 02:34:15.315919 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User: tempest-TestShelveInstance-2138144306-project Aug 08 02:34:15.315919 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Id: default Aug 08 02:34:15.315919 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Name: Default Aug 08 02:34:15.315919 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Id: 8be4cb4f1c594f42825eb40985a29cac Aug 08 02:34:15.315919 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Name: tempest-TestShelveInstance-2138144306-project Aug 08 02:34:15.315919 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: method: > Aug 08 02:34:15.315919 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: exception: Instance 077e7935-3361-4ff5-9367-ba162efac09c could not be found. Aug 08 02:34:15.315919 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: action_args: {'id': '077e7935-3361-4ff5-9367-ba162efac09c'} Aug 08 02:34:15.317463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:34:15.317463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:34:15.317463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 08 02:34:15.317463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 08 02:34:15.317463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 08 02:34:15.317463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 08 02:34:15.317463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 08 02:34:15.317463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 08 02:34:15.317463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 08 02:34:15.317463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 08 02:34:15.317463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 077e7935-3361-4ff5-9367-ba162efac09c could not be found. Aug 08 02:34:15.317463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:34:15.317463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:34:15.317463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:34:15.317463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:34:15.317463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:34:15.317463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:34:15.317463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:34:15.317463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:34:15.317463 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:34:15.319431 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:34:15.319431 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 08 02:34:15.319431 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 08 02:34:15.319431 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 08 02:34:15.319431 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 08 02:34:15.319431 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 08 02:34:15.319431 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:34:15.319431 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 077e7935-3361-4ff5-9367-ba162efac09c could not be found. Aug 08 02:34:15.319431 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:34:15.319431 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:34:15.319431 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:34:15.319431 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:34:15.319431 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:34:15.319431 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:34:15.319431 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:34:15.319431 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:34:15.319431 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 077e7935-3361-4ff5-9367-ba162efac09c could not be found. Aug 08 02:34:15.319431 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:34:15.320930 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-a23b07e8-c8d7-4642-b376-bf7cd4b05e96 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Returning 404 to user: Instance 077e7935-3361-4ff5-9367-ba162efac09c could not be found. {{(pid=118352) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:34:15.320930 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-a23b07e8-c8d7-4642-b376-bf7cd4b05e96 tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c" status: 404 len: 111 microversion: 2.1 time: 0.058003 Aug 08 02:34:15.320930 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 511/1012] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:15 2022] GET /compute/v2.1/servers/077e7935-3361-4ff5-9367-ba162efac09c => generated 111 bytes in 60 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 08 02:34:15.557639 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-ac0b51a8-23fc-413d-a48f-5f8d6d449ee6 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:15.558838 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-ac0b51a8-23fc-413d-a48f-5f8d6d449ee6 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] [instance: d1c59bd2-0519-4579-85f3-532b1b8400a8] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:15.579416 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ac0b51a8-23fc-413d-a48f-5f8d6d449ee6 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:15.608916 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ac0b51a8-23fc-413d-a48f-5f8d6d449ee6 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:15.609668 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ac0b51a8-23fc-413d-a48f-5f8d6d449ee6 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.017s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:15.610725 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ac0b51a8-23fc-413d-a48f-5f8d6d449ee6 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:15.611401 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ac0b51a8-23fc-413d-a48f-5f8d6d449ee6 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:15.614192 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ac0b51a8-23fc-413d-a48f-5f8d6d449ee6 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:15.874156 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-67ba1549-b25f-4f34-839b-960e179b11ec tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] Calling method '.version_select at 0x7efc7414a820>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:15.978046 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-ac0b51a8-23fc-413d-a48f-5f8d6d449ee6 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d75fc1643de49c680b5cb6b977fdb0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c30c9b3f01e740c2a329123736eb2e5e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:15.982606 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-67ba1549-b25f-4f34-839b-960e179b11ec tempest-TestShelveInstance-2138144306 tempest-TestShelveInstance-2138144306-project] 10.222.0.27 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-556747333" status: 202 len: 0 microversion: 2.1 time: 0.112589 Aug 08 02:34:15.983743 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 512/1013] 10.222.0.27 () {58 vars in 1251 bytes} [Mon Aug 8 02:34:15 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-556747333 => generated 0 bytes in 114 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 08 02:34:15.998442 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-ac0b51a8-23fc-413d-a48f-5f8d6d449ee6 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] 10.222.0.27 "GET /compute/v2.1/servers/d1c59bd2-0519-4579-85f3-532b1b8400a8" status: 200 len: 1464 microversion: 2.1 time: 0.444927 Aug 08 02:34:16.005492 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 502/1014] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:15 2022] GET /compute/v2.1/servers/d1c59bd2-0519-4579-85f3-532b1b8400a8 => generated 1464 bytes in 453 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:16.106487 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-e2109bbf-9856-4807-9f31-a4426b69d3b5 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:16.107000 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-e2109bbf-9856-4807-9f31-a4426b69d3b5 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] [instance: d1c59bd2-0519-4579-85f3-532b1b8400a8] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:16.129951 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e2109bbf-9856-4807-9f31-a4426b69d3b5 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:16.130287 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e2109bbf-9856-4807-9f31-a4426b69d3b5 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:16.133060 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e2109bbf-9856-4807-9f31-a4426b69d3b5 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:16.136323 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e2109bbf-9856-4807-9f31-a4426b69d3b5 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:16.137730 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e2109bbf-9856-4807-9f31-a4426b69d3b5 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:16.140151 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e2109bbf-9856-4807-9f31-a4426b69d3b5 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:16.214409 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-e2109bbf-9856-4807-9f31-a4426b69d3b5 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] [instance: d1c59bd2-0519-4579-85f3-532b1b8400a8] Going to try to terminate instance {{(pid=118352) delete /opt/stack/nova/nova/compute/api.py:2666}} Aug 08 02:34:16.922852 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-e2109bbf-9856-4807-9f31-a4426b69d3b5 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] 10.222.0.27 "DELETE /compute/v2.1/servers/d1c59bd2-0519-4579-85f3-532b1b8400a8" status: 204 len: 0 microversion: 2.1 time: 0.821141 Aug 08 02:34:16.929014 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 513/1015] 10.222.0.27 () {58 vars in 1239 bytes} [Mon Aug 8 02:34:16 2022] DELETE /compute/v2.1/servers/d1c59bd2-0519-4579-85f3-532b1b8400a8 => generated 0 bytes in 825 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 08 02:34:16.937213 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-672385cb-2680-4628-ac45-660d5b4af73a tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:16.938194 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-672385cb-2680-4628-ac45-660d5b4af73a tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] [instance: d1c59bd2-0519-4579-85f3-532b1b8400a8] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:16.968384 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-672385cb-2680-4628-ac45-660d5b4af73a tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:16.974081 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-672385cb-2680-4628-ac45-660d5b4af73a tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.015s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:16.974878 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-672385cb-2680-4628-ac45-660d5b4af73a tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:16.976389 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-672385cb-2680-4628-ac45-660d5b4af73a tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:16.980264 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-672385cb-2680-4628-ac45-660d5b4af73a tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:16.981181 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-672385cb-2680-4628-ac45-660d5b4af73a tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:17.377244 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-672385cb-2680-4628-ac45-660d5b4af73a tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d75fc1643de49c680b5cb6b977fdb0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c30c9b3f01e740c2a329123736eb2e5e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:17.391954 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-672385cb-2680-4628-ac45-660d5b4af73a tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] 10.222.0.27 "GET /compute/v2.1/servers/d1c59bd2-0519-4579-85f3-532b1b8400a8" status: 200 len: 1470 microversion: 2.1 time: 0.454008 Aug 08 02:34:17.395219 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 503/1016] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:16 2022] GET /compute/v2.1/servers/d1c59bd2-0519-4579-85f3-532b1b8400a8 => generated 1470 bytes in 465 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:17.728211 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-01431632-e09e-4764-9ec9-f8ac11868f7e tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Action: 'create', calling method: .version_select at 0x7efc74081c10>, body: {"keypair": {"name": "tempest-TestVolumeBootPattern-1683228822", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBENTMgJwV5dmAAkV2h8rp3D0Nft2xG9UuZofOJHlB+dmLDCOi+/VLNlHeF2Ow+fQi8lQx/2/qrp7Lo0eOftT60xqf5KPe3tUEG4Dyk9b64QA9DyL1a8LiAt0gLgclHbQbg=="}} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:34:17.749011 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.quota [None req-01431632-e09e-4764-9ec9-f8ac11868f7e tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Getting quotas for project 4e2c1907fad4499aae599d49e1d2bc80. Resources: {'key_pairs'} {{(pid=118352) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 08 02:34:17.753363 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.quota [None req-01431632-e09e-4764-9ec9-f8ac11868f7e tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Getting quotas for user b92e9b1d8c96434086055f58b119ed57 and project 4e2c1907fad4499aae599d49e1d2bc80. Resources: {'key_pairs'} {{(pid=118352) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 08 02:34:17.857477 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-01431632-e09e-4764-9ec9-f8ac11868f7e tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "POST /compute/v2.1/os-keypairs" status: 200 len: 400 microversion: 2.1 time: 0.417824 Aug 08 02:34:17.857998 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 514/1017] 10.222.0.27 () {60 vars in 1159 bytes} [Mon Aug 8 02:34:17 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 420 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:34:18.417707 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-f330ad4b-c6c1-4c7e-a698-1a3e07d5acb1 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:18.418424 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-f330ad4b-c6c1-4c7e-a698-1a3e07d5acb1 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] [instance: d1c59bd2-0519-4579-85f3-532b1b8400a8] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:18.427255 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f330ad4b-c6c1-4c7e-a698-1a3e07d5acb1 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:18.428049 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f330ad4b-c6c1-4c7e-a698-1a3e07d5acb1 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:18.429200 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f330ad4b-c6c1-4c7e-a698-1a3e07d5acb1 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:18.430588 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f330ad4b-c6c1-4c7e-a698-1a3e07d5acb1 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:18.431226 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f330ad4b-c6c1-4c7e-a698-1a3e07d5acb1 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:18.432328 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f330ad4b-c6c1-4c7e-a698-1a3e07d5acb1 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:18.753403 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-f330ad4b-c6c1-4c7e-a698-1a3e07d5acb1 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d75fc1643de49c680b5cb6b977fdb0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c30c9b3f01e740c2a329123736eb2e5e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:18.776938 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-f330ad4b-c6c1-4c7e-a698-1a3e07d5acb1 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] 10.222.0.27 "GET /compute/v2.1/servers/d1c59bd2-0519-4579-85f3-532b1b8400a8" status: 200 len: 1470 microversion: 2.1 time: 0.362942 Aug 08 02:34:18.778199 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 504/1018] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:18 2022] GET /compute/v2.1/servers/d1c59bd2-0519-4579-85f3-532b1b8400a8 => generated 1470 bytes in 365 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:19.359857 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [req-e8b5eff1-d64e-4a6b-aae8-6729f449542a req-b80ce152-f77f-48f8-8fe4-6ba6963ae6be service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d1c59bd2-0519-4579-85f3-532b1b8400a8", "name": "network-vif-deleted", "tag": "419e2ca2-827d-458a-8c88-d8c3a45cfcae"}]} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:34:19.378214 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-e8b5eff1-d64e-4a6b-aae8-6729f449542a req-b80ce152-f77f-48f8-8fe4-6ba6963ae6be service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:19.379020 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-e8b5eff1-d64e-4a6b-aae8-6729f449542a req-b80ce152-f77f-48f8-8fe4-6ba6963ae6be service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:19.379839 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-e8b5eff1-d64e-4a6b-aae8-6729f449542a req-b80ce152-f77f-48f8-8fe4-6ba6963ae6be service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:19.399235 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.compute.server_external_events [req-e8b5eff1-d64e-4a6b-aae8-6729f449542a req-b80ce152-f77f-48f8-8fe4-6ba6963ae6be service nova] Creating event network-vif-deleted:419e2ca2-827d-458a-8c88-d8c3a45cfcae for instance d1c59bd2-0519-4579-85f3-532b1b8400a8 on hv-cinder-74014 Aug 08 02:34:19.407764 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [req-e8b5eff1-d64e-4a6b-aae8-6729f449542a req-b80ce152-f77f-48f8-8fe4-6ba6963ae6be service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.052631 Aug 08 02:34:19.408706 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 515/1019] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:34:19 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:34:19.799021 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-3f61c4cf-02d8-4758-b7c2-f2670963ad74 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:19.801228 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-3f61c4cf-02d8-4758-b7c2-f2670963ad74 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] [instance: d1c59bd2-0519-4579-85f3-532b1b8400a8] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:19.815271 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3f61c4cf-02d8-4758-b7c2-f2670963ad74 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:19.815775 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3f61c4cf-02d8-4758-b7c2-f2670963ad74 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:19.819877 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3f61c4cf-02d8-4758-b7c2-f2670963ad74 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:19.820708 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3f61c4cf-02d8-4758-b7c2-f2670963ad74 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:19.821195 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3f61c4cf-02d8-4758-b7c2-f2670963ad74 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:19.821644 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3f61c4cf-02d8-4758-b7c2-f2670963ad74 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:19.896874 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3f61c4cf-02d8-4758-b7c2-f2670963ad74 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:19.897370 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3f61c4cf-02d8-4758-b7c2-f2670963ad74 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:19.900426 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3f61c4cf-02d8-4758-b7c2-f2670963ad74 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:19.900426 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.objects.instance [None req-3f61c4cf-02d8-4758-b7c2-f2670963ad74 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lazy-loading 'fault' on Instance uuid d1c59bd2-0519-4579-85f3-532b1b8400a8 {{(pid=118351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 08 02:34:19.970250 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-3f61c4cf-02d8-4758-b7c2-f2670963ad74 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d75fc1643de49c680b5cb6b977fdb0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c30c9b3f01e740c2a329123736eb2e5e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:19.986349 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-3f61c4cf-02d8-4758-b7c2-f2670963ad74 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] 10.222.0.27 "GET /compute/v2.1/servers/d1c59bd2-0519-4579-85f3-532b1b8400a8" status: 200 len: 1304 microversion: 2.1 time: 0.191304 Aug 08 02:34:19.988603 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 505/1020] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:19 2022] GET /compute/v2.1/servers/d1c59bd2-0519-4579-85f3-532b1b8400a8 => generated 1304 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:21.012615 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-609375e1-bfb1-40ec-8837-f353b9815149 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:21.013448 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-609375e1-bfb1-40ec-8837-f353b9815149 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] [instance: d1c59bd2-0519-4579-85f3-532b1b8400a8] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:21.023012 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-609375e1-bfb1-40ec-8837-f353b9815149 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:21.024283 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-609375e1-bfb1-40ec-8837-f353b9815149 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:21.025115 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-609375e1-bfb1-40ec-8837-f353b9815149 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:21.025939 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-609375e1-bfb1-40ec-8837-f353b9815149 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:21.026508 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-609375e1-bfb1-40ec-8837-f353b9815149 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:21.027238 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-609375e1-bfb1-40ec-8837-f353b9815149 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:21.049857 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.wsgi [None req-609375e1-bfb1-40ec-8837-f353b9815149 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] HTTP exception thrown: Instance d1c59bd2-0519-4579-85f3-532b1b8400a8 could not be found. Aug 08 02:34:21.051928 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-609375e1-bfb1-40ec-8837-f353b9815149 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Request method failure captured: Aug 08 02:34:21.051928 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: request: GET /compute/v2.1/servers/d1c59bd2-0519-4579-85f3-532b1b8400a8 HTTP/1.1 Aug 08 02:34:21.051928 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept: application/json Aug 08 02:34:21.051928 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept-Encoding: identity Aug 08 02:34:21.051928 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Connection: close Aug 08 02:34:21.051928 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Length: 0 Aug 08 02:34:21.051928 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Type: application/json Aug 08 02:34:21.051928 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Host: 10.222.0.27 Aug 08 02:34:21.051928 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Openstack-System-Scope: None Aug 08 02:34:21.051928 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: User-Agent: python-urllib3/1.26.11 Aug 08 02:34:21.051928 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Auth-Token: gAAAAABi8HYUPG1dOi8rLGCEb8WvgF8h1BXtJDDbJPQD9hjQ9_ODzYR1wNkbX9LSBGwTXYzhlfaUxJLxleZhYQ_loZvdoiGJhzlnYwvv3K6JuC0wj_vV8CDw0_BfaFA4W1ZJFOA6mOydRwokKlLXmFwljECbn9LYnyMBYRLzCTJ-vBfBo7UQBDw Aug 08 02:34:21.051928 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Id: None Aug 08 02:34:21.051928 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Name: None Aug 08 02:34:21.051928 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Identity-Status: Confirmed Aug 08 02:34:21.051928 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Is-Admin-Project: True Aug 08 02:34:21.051928 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Id: default Aug 08 02:34:21.051928 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Name: Default Aug 08 02:34:21.051928 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Id: c30c9b3f01e740c2a329123736eb2e5e Aug 08 02:34:21.051928 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Name: tempest-VolumesNegativeTest-1254871759 Aug 08 02:34:21.051928 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Role: reader,member Aug 08 02:34:21.051928 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Roles: reader,member Aug 08 02:34:21.053644 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/c30c9b3f01e740c2a329123736eb2e5e"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/c30c9b3f01e740c2a329123736eb2e5e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_c30c9b3f01e740c2a329123736eb2e5e", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/c30c9b3f01e740c2a329123736eb2e5e"}]}] Aug 08 02:34:21.053644 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant: tempest-VolumesNegativeTest-1254871759 Aug 08 02:34:21.053644 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Id: c30c9b3f01e740c2a329123736eb2e5e Aug 08 02:34:21.053644 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Name: tempest-VolumesNegativeTest-1254871759 Aug 08 02:34:21.053644 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User: tempest-VolumesNegativeTest-1254871759-project Aug 08 02:34:21.053644 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Id: default Aug 08 02:34:21.053644 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Name: Default Aug 08 02:34:21.053644 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Id: 0d75fc1643de49c680b5cb6b977fdb0c Aug 08 02:34:21.053644 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Name: tempest-VolumesNegativeTest-1254871759-project Aug 08 02:34:21.053644 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: method: > Aug 08 02:34:21.053644 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: exception: Instance d1c59bd2-0519-4579-85f3-532b1b8400a8 could not be found. Aug 08 02:34:21.053644 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: action_args: {'id': 'd1c59bd2-0519-4579-85f3-532b1b8400a8'} Aug 08 02:34:21.054467 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:34:21.054467 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:34:21.054467 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 08 02:34:21.054467 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 08 02:34:21.054467 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 08 02:34:21.054467 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 08 02:34:21.054467 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 08 02:34:21.054467 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 08 02:34:21.054467 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 08 02:34:21.054467 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 08 02:34:21.054467 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d1c59bd2-0519-4579-85f3-532b1b8400a8 could not be found. Aug 08 02:34:21.054467 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:34:21.054467 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:34:21.054467 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:34:21.054467 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:34:21.054467 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:34:21.054467 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:34:21.054467 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:34:21.054467 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:34:21.054467 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:34:21.055718 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:34:21.055718 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 08 02:34:21.055718 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 08 02:34:21.055718 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 08 02:34:21.055718 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 08 02:34:21.055718 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 08 02:34:21.055718 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:34:21.055718 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d1c59bd2-0519-4579-85f3-532b1b8400a8 could not be found. Aug 08 02:34:21.055718 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:34:21.055718 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:34:21.055718 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:34:21.055718 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:34:21.055718 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:34:21.055718 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:34:21.055718 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:34:21.055718 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:34:21.055718 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d1c59bd2-0519-4579-85f3-532b1b8400a8 could not be found. Aug 08 02:34:21.055718 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:34:21.059807 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-609375e1-bfb1-40ec-8837-f353b9815149 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] Returning 404 to user: Instance d1c59bd2-0519-4579-85f3-532b1b8400a8 could not be found. {{(pid=118352) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:34:21.060767 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-609375e1-bfb1-40ec-8837-f353b9815149 tempest-VolumesNegativeTest-1254871759 tempest-VolumesNegativeTest-1254871759-project] 10.222.0.27 "GET /compute/v2.1/servers/d1c59bd2-0519-4579-85f3-532b1b8400a8" status: 404 len: 111 microversion: 2.1 time: 0.052670 Aug 08 02:34:21.061919 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 516/1021] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:21 2022] GET /compute/v2.1/servers/d1c59bd2-0519-4579-85f3-532b1b8400a8 => generated 111 bytes in 54 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 08 02:34:27.726148 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-668ef0ab-04ec-4e60-b59b-ccddf4ff64be tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Action: 'create', calling method: .version_select at 0x7efc740bf940>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-608953587", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNeaLjFwnej9MkGJpYBCeY53VgkBdvv9Th1guYav6lszGIKRPjQ70AGgPH+Td1KR2nrfKzlP3l5sj37jqeuvXBTzMPALDtA+rAtVHATOkJ9hEPVv0Mmk1TOGTLQbv0qgEg=="}} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:34:27.744530 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.quota [None req-668ef0ab-04ec-4e60-b59b-ccddf4ff64be tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Getting quotas for project dc5ea7a1b6a54cd2af5d648f09aaed63. Resources: {'key_pairs'} {{(pid=118351) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 08 02:34:27.749154 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.quota [None req-668ef0ab-04ec-4e60-b59b-ccddf4ff64be tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Getting quotas for user f5975379433a4e31a242764f58954f43 and project dc5ea7a1b6a54cd2af5d648f09aaed63. Resources: {'key_pairs'} {{(pid=118351) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 08 02:34:27.857199 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-668ef0ab-04ec-4e60-b59b-ccddf4ff64be tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "POST /compute/v2.1/os-keypairs" status: 200 len: 402 microversion: 2.1 time: 0.135935 Aug 08 02:34:27.869078 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 506/1022] 10.222.0.27 () {60 vars in 1159 bytes} [Mon Aug 8 02:34:27 2022] POST /compute/v2.1/os-keypairs => generated 402 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:34:27.918777 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-f7b99985-8de1-4910-87a9-161699314169 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-1546977096", "imageRef": "a1a1a445-1112-402e-a7fb-3bbfcc12197a", "flavorRef": "452", "key_name": "tempest-TestMinimumBasicScenario-608953587", "networks": [{"uuid": "4d09ced2-0f8b-409a-a987-7674e6c24849"}]}} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:34:27.935991 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell0:SELECT=4 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:28.482388 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.network.neutron [None req-f7b99985-8de1-4910-87a9-161699314169 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] validate_networks() for [('4d09ced2-0f8b-409a-a987-7674e6c24849', None, None, None, None, None)] {{(pid=118352) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 08 02:34:28.954053 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-f7b99985-8de1-4910-87a9-161699314169 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Flavor limits 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 08 02:34:28.955101 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-f7b99985-8de1-4910-87a9-161699314169 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Image limits 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 08 02:34:28.955767 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-f7b99985-8de1-4910-87a9-161699314169 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Flavor pref 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 08 02:34:28.960849 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-f7b99985-8de1-4910-87a9-161699314169 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Image pref 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 08 02:34:28.961637 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-f7b99985-8de1-4910-87a9-161699314169 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 08 02:34:28.977008 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f7b99985-8de1-4910-87a9-161699314169 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:28.977500 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f7b99985-8de1-4910-87a9-161699314169 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:28.978040 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f7b99985-8de1-4910-87a9-161699314169 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:28.978795 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f7b99985-8de1-4910-87a9-161699314169 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:28.979274 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f7b99985-8de1-4910-87a9-161699314169 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:28.979697 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f7b99985-8de1-4910-87a9-161699314169 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:28.997822 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.quota [None req-f7b99985-8de1-4910-87a9-161699314169 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Getting quotas for project dc5ea7a1b6a54cd2af5d648f09aaed63. Resources: {'ram', 'cores', 'instances'} {{(pid=118352) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 08 02:34:29.002656 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.quota [None req-f7b99985-8de1-4910-87a9-161699314169 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Getting quotas for user f5975379433a4e31a242764f58954f43 and project dc5ea7a1b6a54cd2af5d648f09aaed63. Resources: {'ram', 'cores', 'instances'} {{(pid=118352) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 08 02:34:29.014854 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-f7b99985-8de1-4910-87a9-161699314169 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Going to run 1 instances... {{(pid=118352) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Aug 08 02:34:29.017224 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-f7b99985-8de1-4910-87a9-161699314169 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] [instance: 0e099ac0-c453-43b0-8bc1-b4c162d483a5] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a1a1a445-1112-402e-a7fb-3bbfcc12197a',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118352) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Aug 08 02:34:29.203119 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-f7b99985-8de1-4910-87a9-161699314169 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.288457 Aug 08 02:34:29.203667 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 517/1023] 10.222.0.27 () {60 vars in 1147 bytes} [Mon Aug 8 02:34:27 2022] POST /compute/v2.1/servers => generated 380 bytes in 1290 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 08 02:34:29.218922 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-376389b3-245e-4d6f-b44e-892e27caec0a tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:29.218922 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-376389b3-245e-4d6f-b44e-892e27caec0a tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] [instance: 0e099ac0-c453-43b0-8bc1-b4c162d483a5] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:29.281013 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-376389b3-245e-4d6f-b44e-892e27caec0a tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5975379433a4e31a242764f58954f43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc5ea7a1b6a54cd2af5d648f09aaed63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:29.292749 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-376389b3-245e-4d6f-b44e-892e27caec0a tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5" status: 200 len: 1258 microversion: 2.1 time: 0.080580 Aug 08 02:34:29.293570 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 507/1024] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:29 2022] GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5 => generated 1258 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:30.315836 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-7b4fcc8e-c345-47cb-b9d0-29ea1a58b354 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:30.316460 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-7b4fcc8e-c345-47cb-b9d0-29ea1a58b354 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] [instance: 0e099ac0-c453-43b0-8bc1-b4c162d483a5] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:30.376833 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-7b4fcc8e-c345-47cb-b9d0-29ea1a58b354 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5975379433a4e31a242764f58954f43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc5ea7a1b6a54cd2af5d648f09aaed63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:30.384220 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-7b4fcc8e-c345-47cb-b9d0-29ea1a58b354 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5" status: 200 len: 1258 microversion: 2.1 time: 0.073128 Aug 08 02:34:30.384825 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 518/1025] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:30 2022] GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5 => generated 1258 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:31.405876 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-7c493476-c67e-45c4-915b-5a3018920855 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:31.406622 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-7c493476-c67e-45c4-915b-5a3018920855 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] [instance: 0e099ac0-c453-43b0-8bc1-b4c162d483a5] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:31.408652 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell1:SELECT=88,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:31.414669 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7c493476-c67e-45c4-915b-5a3018920855 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:31.417595 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7c493476-c67e-45c4-915b-5a3018920855 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:31.417595 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7c493476-c67e-45c4-915b-5a3018920855 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:31.417595 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7c493476-c67e-45c4-915b-5a3018920855 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:31.417595 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7c493476-c67e-45c4-915b-5a3018920855 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:31.418261 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7c493476-c67e-45c4-915b-5a3018920855 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:31.486072 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-7c493476-c67e-45c4-915b-5a3018920855 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5975379433a4e31a242764f58954f43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc5ea7a1b6a54cd2af5d648f09aaed63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:31.494648 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-7c493476-c67e-45c4-915b-5a3018920855 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5" status: 200 len: 1310 microversion: 2.1 time: 0.092667 Aug 08 02:34:31.495213 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 508/1026] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:31 2022] GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5 => generated 1310 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:32.516142 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-af3044c1-e8bd-4212-91f0-2494cca811a1 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:32.516784 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-af3044c1-e8bd-4212-91f0-2494cca811a1 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] [instance: 0e099ac0-c453-43b0-8bc1-b4c162d483a5] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:32.519853 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_api:SELECT=71,nova_api:DELETE=1,nova_api:UPDATE=1,nova_api:INSERT=4 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:32.524893 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-af3044c1-e8bd-4212-91f0-2494cca811a1 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:32.525681 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-af3044c1-e8bd-4212-91f0-2494cca811a1 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:32.526392 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-af3044c1-e8bd-4212-91f0-2494cca811a1 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:32.527352 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-af3044c1-e8bd-4212-91f0-2494cca811a1 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:32.528106 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-af3044c1-e8bd-4212-91f0-2494cca811a1 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:32.528543 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-af3044c1-e8bd-4212-91f0-2494cca811a1 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:32.532987 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell1:SELECT=71,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:32.594685 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-af3044c1-e8bd-4212-91f0-2494cca811a1 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5975379433a4e31a242764f58954f43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc5ea7a1b6a54cd2af5d648f09aaed63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:32.603503 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-af3044c1-e8bd-4212-91f0-2494cca811a1 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5" status: 200 len: 1316 microversion: 2.1 time: 0.091070 Aug 08 02:34:32.604432 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 519/1027] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:32 2022] GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5 => generated 1316 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:33.630841 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-7808c326-1995-4ec5-b3ba-77248fa76e89 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:33.631543 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-7808c326-1995-4ec5-b3ba-77248fa76e89 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] [instance: 0e099ac0-c453-43b0-8bc1-b4c162d483a5] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:33.639618 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_api:SELECT=70,nova_api:INSERT=4,nova_api:UPDATE=1 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:33.643623 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7808c326-1995-4ec5-b3ba-77248fa76e89 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:33.646008 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7808c326-1995-4ec5-b3ba-77248fa76e89 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:33.646008 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7808c326-1995-4ec5-b3ba-77248fa76e89 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:33.646238 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7808c326-1995-4ec5-b3ba-77248fa76e89 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:33.646675 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7808c326-1995-4ec5-b3ba-77248fa76e89 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:33.647102 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7808c326-1995-4ec5-b3ba-77248fa76e89 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:33.871032 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-7808c326-1995-4ec5-b3ba-77248fa76e89 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5975379433a4e31a242764f58954f43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc5ea7a1b6a54cd2af5d648f09aaed63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:33.888339 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-7808c326-1995-4ec5-b3ba-77248fa76e89 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5" status: 200 len: 1358 microversion: 2.1 time: 0.256189 Aug 08 02:34:33.888339 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 509/1028] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:33 2022] GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5 => generated 1358 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:33.969754 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [req-edc9b4ac-d47e-4f9a-8dac-916c54357956 req-b18b7eca-1c95-4d78-b8ca-abc25962d7a2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0e099ac0-c453-43b0-8bc1-b4c162d483a5", "tag": "35cd469b-968e-46d3-8566-8026ce405507"}]} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:34:33.983523 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-edc9b4ac-d47e-4f9a-8dac-916c54357956 req-b18b7eca-1c95-4d78-b8ca-abc25962d7a2 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:33.984323 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-edc9b4ac-d47e-4f9a-8dac-916c54357956 req-b18b7eca-1c95-4d78-b8ca-abc25962d7a2 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:33.986691 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-edc9b4ac-d47e-4f9a-8dac-916c54357956 req-b18b7eca-1c95-4d78-b8ca-abc25962d7a2 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:34.013672 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.compute.server_external_events [req-edc9b4ac-d47e-4f9a-8dac-916c54357956 req-b18b7eca-1c95-4d78-b8ca-abc25962d7a2 service nova] Creating event network-changed:35cd469b-968e-46d3-8566-8026ce405507 for instance 0e099ac0-c453-43b0-8bc1-b4c162d483a5 on hv-cinder-74014 Aug 08 02:34:34.027583 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [req-edc9b4ac-d47e-4f9a-8dac-916c54357956 req-b18b7eca-1c95-4d78-b8ca-abc25962d7a2 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.066401 Aug 08 02:34:34.028705 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 520/1029] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:34:33 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:34:34.906131 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-110c1133-cb98-420d-82da-670f9e9c92bd tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:34.906770 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-110c1133-cb98-420d-82da-670f9e9c92bd tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] [instance: 0e099ac0-c453-43b0-8bc1-b4c162d483a5] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:34.914262 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-110c1133-cb98-420d-82da-670f9e9c92bd tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:34.914756 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-110c1133-cb98-420d-82da-670f9e9c92bd tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:34.915192 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-110c1133-cb98-420d-82da-670f9e9c92bd tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:34.915897 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-110c1133-cb98-420d-82da-670f9e9c92bd tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:34.916352 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-110c1133-cb98-420d-82da-670f9e9c92bd tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:34.916802 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-110c1133-cb98-420d-82da-670f9e9c92bd tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:35.078409 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-110c1133-cb98-420d-82da-670f9e9c92bd tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5975379433a4e31a242764f58954f43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc5ea7a1b6a54cd2af5d648f09aaed63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:35.086490 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-110c1133-cb98-420d-82da-670f9e9c92bd tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5" status: 200 len: 1358 microversion: 2.1 time: 0.184886 Aug 08 02:34:35.087064 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 510/1030] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:34 2022] GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5 => generated 1358 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:36.109227 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-0c92bf54-2191-4287-a735-182594dd6690 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:36.110055 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-0c92bf54-2191-4287-a735-182594dd6690 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] [instance: 0e099ac0-c453-43b0-8bc1-b4c162d483a5] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:36.118135 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0c92bf54-2191-4287-a735-182594dd6690 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:36.118780 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0c92bf54-2191-4287-a735-182594dd6690 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:36.119305 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0c92bf54-2191-4287-a735-182594dd6690 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:36.120179 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0c92bf54-2191-4287-a735-182594dd6690 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:36.120631 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0c92bf54-2191-4287-a735-182594dd6690 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:36.121197 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0c92bf54-2191-4287-a735-182594dd6690 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:36.276065 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-0c92bf54-2191-4287-a735-182594dd6690 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5975379433a4e31a242764f58954f43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc5ea7a1b6a54cd2af5d648f09aaed63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:36.284558 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-0c92bf54-2191-4287-a735-182594dd6690 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5" status: 200 len: 1358 microversion: 2.1 time: 0.179376 Aug 08 02:34:36.285737 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 521/1031] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:36 2022] GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5 => generated 1358 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:37.307562 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-05de9375-c7c5-4073-a3fe-6258f30a0410 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:37.308216 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-05de9375-c7c5-4073-a3fe-6258f30a0410 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] [instance: 0e099ac0-c453-43b0-8bc1-b4c162d483a5] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:37.314933 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-05de9375-c7c5-4073-a3fe-6258f30a0410 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:37.315419 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-05de9375-c7c5-4073-a3fe-6258f30a0410 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:37.317137 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-05de9375-c7c5-4073-a3fe-6258f30a0410 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:37.317850 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-05de9375-c7c5-4073-a3fe-6258f30a0410 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:37.318307 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-05de9375-c7c5-4073-a3fe-6258f30a0410 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:37.318733 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-05de9375-c7c5-4073-a3fe-6258f30a0410 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:37.490048 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-05de9375-c7c5-4073-a3fe-6258f30a0410 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5975379433a4e31a242764f58954f43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc5ea7a1b6a54cd2af5d648f09aaed63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:37.490048 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-05de9375-c7c5-4073-a3fe-6258f30a0410 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5" status: 200 len: 1358 microversion: 2.1 time: 0.174230 Aug 08 02:34:37.490048 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 511/1032] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:37 2022] GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5 => generated 1358 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:38.510146 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-474bb6c7-99d6-487e-9dc1-7515e8d3a154 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:38.510726 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-474bb6c7-99d6-487e-9dc1-7515e8d3a154 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] [instance: 0e099ac0-c453-43b0-8bc1-b4c162d483a5] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:38.517296 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-474bb6c7-99d6-487e-9dc1-7515e8d3a154 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:38.517779 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-474bb6c7-99d6-487e-9dc1-7515e8d3a154 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:38.518216 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-474bb6c7-99d6-487e-9dc1-7515e8d3a154 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:38.518905 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-474bb6c7-99d6-487e-9dc1-7515e8d3a154 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:38.519908 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-474bb6c7-99d6-487e-9dc1-7515e8d3a154 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:38.520345 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-474bb6c7-99d6-487e-9dc1-7515e8d3a154 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:38.672912 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-474bb6c7-99d6-487e-9dc1-7515e8d3a154 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5975379433a4e31a242764f58954f43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc5ea7a1b6a54cd2af5d648f09aaed63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:38.681398 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-474bb6c7-99d6-487e-9dc1-7515e8d3a154 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5" status: 200 len: 1358 microversion: 2.1 time: 0.175161 Aug 08 02:34:38.682196 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 522/1033] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:38 2022] GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5 => generated 1358 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:39.704000 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-351aedcb-1a01-4fd2-b175-61057b5f20ef tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:39.704645 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-351aedcb-1a01-4fd2-b175-61057b5f20ef tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] [instance: 0e099ac0-c453-43b0-8bc1-b4c162d483a5] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:39.711547 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell0:SELECT=6 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:39.713067 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-351aedcb-1a01-4fd2-b175-61057b5f20ef tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:39.713760 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-351aedcb-1a01-4fd2-b175-61057b5f20ef tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:39.714411 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-351aedcb-1a01-4fd2-b175-61057b5f20ef tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:39.715330 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-351aedcb-1a01-4fd2-b175-61057b5f20ef tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:39.715955 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-351aedcb-1a01-4fd2-b175-61057b5f20ef tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:39.716554 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-351aedcb-1a01-4fd2-b175-61057b5f20ef tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:39.885627 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-351aedcb-1a01-4fd2-b175-61057b5f20ef tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5975379433a4e31a242764f58954f43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc5ea7a1b6a54cd2af5d648f09aaed63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:39.893743 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-351aedcb-1a01-4fd2-b175-61057b5f20ef tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5" status: 200 len: 1358 microversion: 2.1 time: 0.193623 Aug 08 02:34:39.895057 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 512/1034] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:39 2022] GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5 => generated 1358 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:40.643253 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-023ddf10-932d-4aae-81ff-86adee4ca36f tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-1638493810", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1683228822", "security_groups": [{"name": "tempest-secgroup-smoke-1655674036"}], "block_device_mapping_v2": [{"uuid": "8d756c2b-af46-4d5c-a12f-63de84861b78", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "b0ff42d3-c706-479c-b637-24fa43da993a"}]}} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:34:40.653117 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell0:SELECT=2 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:40.654378 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell0:SELECT=8 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:40.762323 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.network.neutron [None req-023ddf10-932d-4aae-81ff-86adee4ca36f tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] validate_networks() for [('b0ff42d3-c706-479c-b637-24fa43da993a', None, None, None, None, None)] {{(pid=118352) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 08 02:34:40.916043 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-18c14a91-3aa8-4d7a-9005-aeeb982bf760 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:40.916720 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-18c14a91-3aa8-4d7a-9005-aeeb982bf760 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] [instance: 0e099ac0-c453-43b0-8bc1-b4c162d483a5] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:40.923649 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-18c14a91-3aa8-4d7a-9005-aeeb982bf760 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:40.924334 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-18c14a91-3aa8-4d7a-9005-aeeb982bf760 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:40.925005 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-18c14a91-3aa8-4d7a-9005-aeeb982bf760 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:40.926201 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-18c14a91-3aa8-4d7a-9005-aeeb982bf760 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:40.926658 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-18c14a91-3aa8-4d7a-9005-aeeb982bf760 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:40.927075 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-18c14a91-3aa8-4d7a-9005-aeeb982bf760 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:41.181898 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-18c14a91-3aa8-4d7a-9005-aeeb982bf760 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5975379433a4e31a242764f58954f43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc5ea7a1b6a54cd2af5d648f09aaed63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:41.198180 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-18c14a91-3aa8-4d7a-9005-aeeb982bf760 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5" status: 200 len: 1358 microversion: 2.1 time: 0.285903 Aug 08 02:34:41.199683 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 513/1035] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:40 2022] GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5 => generated 1358 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:34:41.234613 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-023ddf10-932d-4aae-81ff-86adee4ca36f tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Flavor limits 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 08 02:34:41.235038 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-023ddf10-932d-4aae-81ff-86adee4ca36f tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Image limits 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 08 02:34:41.235511 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-023ddf10-932d-4aae-81ff-86adee4ca36f tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Flavor pref 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 08 02:34:41.235920 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-023ddf10-932d-4aae-81ff-86adee4ca36f tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Image pref 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 08 02:34:41.236324 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-023ddf10-932d-4aae-81ff-86adee4ca36f tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 08 02:34:41.247936 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-023ddf10-932d-4aae-81ff-86adee4ca36f tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:41.248420 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-023ddf10-932d-4aae-81ff-86adee4ca36f tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:41.248899 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-023ddf10-932d-4aae-81ff-86adee4ca36f tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:41.249625 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-023ddf10-932d-4aae-81ff-86adee4ca36f tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:41.250083 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-023ddf10-932d-4aae-81ff-86adee4ca36f tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:41.250512 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-023ddf10-932d-4aae-81ff-86adee4ca36f tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:41.270324 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.quota [None req-023ddf10-932d-4aae-81ff-86adee4ca36f tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Getting quotas for project 4e2c1907fad4499aae599d49e1d2bc80. Resources: {'ram', 'cores', 'instances'} {{(pid=118352) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 08 02:34:41.274666 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.quota [None req-023ddf10-932d-4aae-81ff-86adee4ca36f tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Getting quotas for user b92e9b1d8c96434086055f58b119ed57 and project 4e2c1907fad4499aae599d49e1d2bc80. Resources: {'ram', 'cores', 'instances'} {{(pid=118352) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 08 02:34:41.338275 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-023ddf10-932d-4aae-81ff-86adee4ca36f tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Going to run 1 instances... {{(pid=118352) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Aug 08 02:34:41.340343 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-023ddf10-932d-4aae-81ff-86adee4ca36f tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: f0d7098c-da43-4381-99e9-b4756dde20c8] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='8d756c2b-af46-4d5c-a12f-63de84861b78',volume_size=None,volume_type=None)] {{(pid=118352) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Aug 08 02:34:41.794203 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-023ddf10-932d-4aae-81ff-86adee4ca36f tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.155012 Aug 08 02:34:41.795249 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 523/1036] 10.222.0.27 () {60 vars in 1147 bytes} [Mon Aug 8 02:34:40 2022] POST /compute/v2.1/servers => generated 406 bytes in 1157 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 08 02:34:41.807569 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-a3503920-49fd-4c0c-8ddf-229b4968e054 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:41.808188 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-a3503920-49fd-4c0c-8ddf-229b4968e054 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: f0d7098c-da43-4381-99e9-b4756dde20c8] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:41.865408 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-a3503920-49fd-4c0c-8ddf-229b4968e054 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:41.872370 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-a3503920-49fd-4c0c-8ddf-229b4968e054 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8" status: 200 len: 1095 microversion: 2.1 time: 0.068940 Aug 08 02:34:41.873172 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 514/1037] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:41 2022] GET /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8 => generated 1095 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:42.219718 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-3cc7c5a4-2195-47fc-9053-57ef9d0564ea tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:42.220386 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-3cc7c5a4-2195-47fc-9053-57ef9d0564ea tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] [instance: 0e099ac0-c453-43b0-8bc1-b4c162d483a5] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:42.227310 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3cc7c5a4-2195-47fc-9053-57ef9d0564ea tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:42.227821 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3cc7c5a4-2195-47fc-9053-57ef9d0564ea tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:42.228263 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3cc7c5a4-2195-47fc-9053-57ef9d0564ea tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:42.228988 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3cc7c5a4-2195-47fc-9053-57ef9d0564ea tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:42.229452 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3cc7c5a4-2195-47fc-9053-57ef9d0564ea tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:42.229885 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3cc7c5a4-2195-47fc-9053-57ef9d0564ea tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:42.368295 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-3cc7c5a4-2195-47fc-9053-57ef9d0564ea tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5975379433a4e31a242764f58954f43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc5ea7a1b6a54cd2af5d648f09aaed63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:42.376241 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-3cc7c5a4-2195-47fc-9053-57ef9d0564ea tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5" status: 200 len: 1358 microversion: 2.1 time: 0.160623 Aug 08 02:34:42.376823 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 524/1038] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:42 2022] GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5 => generated 1358 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:42.893711 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-256e67f2-1fa9-45c8-a226-c86a5dd21960 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:42.894326 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-256e67f2-1fa9-45c8-a226-c86a5dd21960 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: f0d7098c-da43-4381-99e9-b4756dde20c8] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:42.901704 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-256e67f2-1fa9-45c8-a226-c86a5dd21960 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:42.902347 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-256e67f2-1fa9-45c8-a226-c86a5dd21960 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:42.902891 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-256e67f2-1fa9-45c8-a226-c86a5dd21960 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:42.903688 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-256e67f2-1fa9-45c8-a226-c86a5dd21960 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:42.904143 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-256e67f2-1fa9-45c8-a226-c86a5dd21960 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:42.904654 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-256e67f2-1fa9-45c8-a226-c86a5dd21960 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:42.969512 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-256e67f2-1fa9-45c8-a226-c86a5dd21960 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:42.977602 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-256e67f2-1fa9-45c8-a226-c86a5dd21960 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8" status: 200 len: 1145 microversion: 2.1 time: 0.087865 Aug 08 02:34:42.982592 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 515/1039] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:42 2022] GET /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8 => generated 1145 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:43.399274 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-7cf60648-87f5-4836-bed2-b5ea6b65803f tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:43.399823 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-7cf60648-87f5-4836-bed2-b5ea6b65803f tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] [instance: 0e099ac0-c453-43b0-8bc1-b4c162d483a5] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:43.406965 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7cf60648-87f5-4836-bed2-b5ea6b65803f tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:43.407515 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7cf60648-87f5-4836-bed2-b5ea6b65803f tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:43.407973 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7cf60648-87f5-4836-bed2-b5ea6b65803f tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:43.408715 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7cf60648-87f5-4836-bed2-b5ea6b65803f tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:43.409226 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7cf60648-87f5-4836-bed2-b5ea6b65803f tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:43.409661 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7cf60648-87f5-4836-bed2-b5ea6b65803f tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:43.542666 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-7cf60648-87f5-4836-bed2-b5ea6b65803f tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5975379433a4e31a242764f58954f43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc5ea7a1b6a54cd2af5d648f09aaed63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:43.550086 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-7cf60648-87f5-4836-bed2-b5ea6b65803f tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5" status: 200 len: 1358 microversion: 2.1 time: 0.156256 Aug 08 02:34:43.550650 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 525/1040] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:43 2022] GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5 => generated 1358 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:43.999296 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-ef66d46f-a4d3-4a85-9a08-630b578bda88 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:43.999913 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-ef66d46f-a4d3-4a85-9a08-630b578bda88 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: f0d7098c-da43-4381-99e9-b4756dde20c8] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:44.006849 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ef66d46f-a4d3-4a85-9a08-630b578bda88 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:44.007481 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ef66d46f-a4d3-4a85-9a08-630b578bda88 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:44.007842 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ef66d46f-a4d3-4a85-9a08-630b578bda88 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:44.008538 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ef66d46f-a4d3-4a85-9a08-630b578bda88 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:44.009089 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ef66d46f-a4d3-4a85-9a08-630b578bda88 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:44.009463 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ef66d46f-a4d3-4a85-9a08-630b578bda88 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:44.074635 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-ef66d46f-a4d3-4a85-9a08-630b578bda88 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:44.083345 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-ef66d46f-a4d3-4a85-9a08-630b578bda88 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8" status: 200 len: 1211 microversion: 2.1 time: 0.087314 Aug 08 02:34:44.083994 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 516/1041] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:43 2022] GET /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8 => generated 1211 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:44.577375 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-b3693737-1412-43a9-9916-6a3840b2f40d tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:44.578110 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-b3693737-1412-43a9-9916-6a3840b2f40d tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] [instance: 0e099ac0-c453-43b0-8bc1-b4c162d483a5] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:44.585804 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b3693737-1412-43a9-9916-6a3840b2f40d tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:44.586369 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b3693737-1412-43a9-9916-6a3840b2f40d tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:44.586831 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b3693737-1412-43a9-9916-6a3840b2f40d tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:44.587890 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b3693737-1412-43a9-9916-6a3840b2f40d tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:44.588410 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b3693737-1412-43a9-9916-6a3840b2f40d tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:44.589052 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b3693737-1412-43a9-9916-6a3840b2f40d tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:44.783538 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-b3693737-1412-43a9-9916-6a3840b2f40d tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5975379433a4e31a242764f58954f43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc5ea7a1b6a54cd2af5d648f09aaed63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:44.798427 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-b3693737-1412-43a9-9916-6a3840b2f40d tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5" status: 200 len: 1358 microversion: 2.1 time: 0.226077 Aug 08 02:34:44.802206 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 526/1042] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:44 2022] GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5 => generated 1358 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:45.104018 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-970bf58c-4bf1-4739-ac44-1e56cf015230 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:45.104587 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-970bf58c-4bf1-4739-ac44-1e56cf015230 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: f0d7098c-da43-4381-99e9-b4756dde20c8] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:45.111347 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-970bf58c-4bf1-4739-ac44-1e56cf015230 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:45.111836 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-970bf58c-4bf1-4739-ac44-1e56cf015230 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:45.112275 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-970bf58c-4bf1-4739-ac44-1e56cf015230 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:45.115967 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-970bf58c-4bf1-4739-ac44-1e56cf015230 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:45.115967 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-970bf58c-4bf1-4739-ac44-1e56cf015230 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:45.115967 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-970bf58c-4bf1-4739-ac44-1e56cf015230 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:45.254175 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-970bf58c-4bf1-4739-ac44-1e56cf015230 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:45.263682 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-970bf58c-4bf1-4739-ac44-1e56cf015230 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8" status: 200 len: 1279 microversion: 2.1 time: 0.163687 Aug 08 02:34:45.264260 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 517/1043] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:45 2022] GET /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8 => generated 1279 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:45.576155 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [req-330dd244-6ffa-4019-9384-f1b52ea02ca4 req-8035881b-72f1-4853-9564-11292d8a2d62 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f0d7098c-da43-4381-99e9-b4756dde20c8", "tag": "80747251-b558-4d89-ba58-3fbb7af11ff4"}]} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:34:45.588172 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-330dd244-6ffa-4019-9384-f1b52ea02ca4 req-8035881b-72f1-4853-9564-11292d8a2d62 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:45.588829 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-330dd244-6ffa-4019-9384-f1b52ea02ca4 req-8035881b-72f1-4853-9564-11292d8a2d62 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:45.588920 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-330dd244-6ffa-4019-9384-f1b52ea02ca4 req-8035881b-72f1-4853-9564-11292d8a2d62 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:45.601529 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.compute.server_external_events [req-330dd244-6ffa-4019-9384-f1b52ea02ca4 req-8035881b-72f1-4853-9564-11292d8a2d62 service nova] Creating event network-changed:80747251-b558-4d89-ba58-3fbb7af11ff4 for instance f0d7098c-da43-4381-99e9-b4756dde20c8 on hv-cinder-74014 Aug 08 02:34:45.606503 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [req-330dd244-6ffa-4019-9384-f1b52ea02ca4 req-8035881b-72f1-4853-9564-11292d8a2d62 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034221 Aug 08 02:34:45.607023 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 527/1044] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:34:45 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:34:45.821645 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-00d0e43f-7fd2-43c8-b88b-a22fc4fbb544 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:45.822384 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-00d0e43f-7fd2-43c8-b88b-a22fc4fbb544 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] [instance: 0e099ac0-c453-43b0-8bc1-b4c162d483a5] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:45.828536 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-00d0e43f-7fd2-43c8-b88b-a22fc4fbb544 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:45.829073 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-00d0e43f-7fd2-43c8-b88b-a22fc4fbb544 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:45.829515 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-00d0e43f-7fd2-43c8-b88b-a22fc4fbb544 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:45.830209 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-00d0e43f-7fd2-43c8-b88b-a22fc4fbb544 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:45.830666 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-00d0e43f-7fd2-43c8-b88b-a22fc4fbb544 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:45.831094 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-00d0e43f-7fd2-43c8-b88b-a22fc4fbb544 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:45.965806 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-00d0e43f-7fd2-43c8-b88b-a22fc4fbb544 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5975379433a4e31a242764f58954f43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc5ea7a1b6a54cd2af5d648f09aaed63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:45.974090 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-00d0e43f-7fd2-43c8-b88b-a22fc4fbb544 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5" status: 200 len: 1358 microversion: 2.1 time: 0.155816 Aug 08 02:34:45.974658 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 518/1045] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:45 2022] GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5 => generated 1358 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:46.285807 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-c198c6a0-7f84-49db-8833-a26822c49d63 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:46.286586 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-c198c6a0-7f84-49db-8833-a26822c49d63 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: f0d7098c-da43-4381-99e9-b4756dde20c8] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:46.292696 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c198c6a0-7f84-49db-8833-a26822c49d63 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:46.293201 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c198c6a0-7f84-49db-8833-a26822c49d63 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:46.293647 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c198c6a0-7f84-49db-8833-a26822c49d63 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:46.294333 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c198c6a0-7f84-49db-8833-a26822c49d63 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:46.294794 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c198c6a0-7f84-49db-8833-a26822c49d63 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:46.295222 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c198c6a0-7f84-49db-8833-a26822c49d63 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:46.429453 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-c198c6a0-7f84-49db-8833-a26822c49d63 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:46.437151 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-c198c6a0-7f84-49db-8833-a26822c49d63 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8" status: 200 len: 1267 microversion: 2.1 time: 0.155482 Aug 08 02:34:46.437690 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 528/1046] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:46 2022] GET /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8 => generated 1267 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:46.685346 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [req-f09c580f-0997-42a6-87b5-2c249ab19a66 req-8265544b-197b-461b-93c1-793b29494b54 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0e099ac0-c453-43b0-8bc1-b4c162d483a5", "name": "network-vif-plugged", "status": "completed", "tag": "35cd469b-968e-46d3-8566-8026ce405507"}]} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:34:46.695821 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-f09c580f-0997-42a6-87b5-2c249ab19a66 req-8265544b-197b-461b-93c1-793b29494b54 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:46.696302 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-f09c580f-0997-42a6-87b5-2c249ab19a66 req-8265544b-197b-461b-93c1-793b29494b54 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:46.696773 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-f09c580f-0997-42a6-87b5-2c249ab19a66 req-8265544b-197b-461b-93c1-793b29494b54 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:46.708041 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.compute.server_external_events [req-f09c580f-0997-42a6-87b5-2c249ab19a66 req-8265544b-197b-461b-93c1-793b29494b54 service nova] Creating event network-vif-plugged:35cd469b-968e-46d3-8566-8026ce405507 for instance 0e099ac0-c453-43b0-8bc1-b4c162d483a5 on hv-cinder-74014 Aug 08 02:34:46.713140 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [req-f09c580f-0997-42a6-87b5-2c249ab19a66 req-8265544b-197b-461b-93c1-793b29494b54 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031389 Aug 08 02:34:46.713711 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 519/1047] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:34:46 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:34:46.995181 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-9bc94626-6a5f-4c44-a39c-28e4eee8a2df tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:46.995909 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-9bc94626-6a5f-4c44-a39c-28e4eee8a2df tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] [instance: 0e099ac0-c453-43b0-8bc1-b4c162d483a5] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:47.002523 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9bc94626-6a5f-4c44-a39c-28e4eee8a2df tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:47.002997 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9bc94626-6a5f-4c44-a39c-28e4eee8a2df tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:47.003431 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9bc94626-6a5f-4c44-a39c-28e4eee8a2df tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:47.004122 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9bc94626-6a5f-4c44-a39c-28e4eee8a2df tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:47.004640 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9bc94626-6a5f-4c44-a39c-28e4eee8a2df tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:47.005135 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9bc94626-6a5f-4c44-a39c-28e4eee8a2df tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:47.138951 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-9bc94626-6a5f-4c44-a39c-28e4eee8a2df tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5975379433a4e31a242764f58954f43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc5ea7a1b6a54cd2af5d648f09aaed63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:47.146537 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-9bc94626-6a5f-4c44-a39c-28e4eee8a2df tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5" status: 200 len: 1358 microversion: 2.1 time: 0.155359 Aug 08 02:34:47.147100 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 529/1048] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:46 2022] GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5 => generated 1358 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:34:47.465527 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-8286b43e-5c6d-4f02-94f8-78aa1f657381 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:47.465527 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-8286b43e-5c6d-4f02-94f8-78aa1f657381 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: f0d7098c-da43-4381-99e9-b4756dde20c8] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:47.467633 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8286b43e-5c6d-4f02-94f8-78aa1f657381 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:47.468115 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8286b43e-5c6d-4f02-94f8-78aa1f657381 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:47.468549 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8286b43e-5c6d-4f02-94f8-78aa1f657381 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:47.469320 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8286b43e-5c6d-4f02-94f8-78aa1f657381 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:47.469777 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8286b43e-5c6d-4f02-94f8-78aa1f657381 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:47.470205 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-8286b43e-5c6d-4f02-94f8-78aa1f657381 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:47.603861 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-8286b43e-5c6d-4f02-94f8-78aa1f657381 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:47.611405 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-8286b43e-5c6d-4f02-94f8-78aa1f657381 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8" status: 200 len: 1267 microversion: 2.1 time: 0.155766 Aug 08 02:34:47.612084 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 520/1049] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:47 2022] GET /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8 => generated 1267 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:48.166941 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-3fbc4205-d1c7-4938-876b-dc91d7201e6b tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:48.167561 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-3fbc4205-d1c7-4938-876b-dc91d7201e6b tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] [instance: 0e099ac0-c453-43b0-8bc1-b4c162d483a5] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:48.174077 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3fbc4205-d1c7-4938-876b-dc91d7201e6b tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:48.174559 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3fbc4205-d1c7-4938-876b-dc91d7201e6b tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:48.174999 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3fbc4205-d1c7-4938-876b-dc91d7201e6b tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:48.175681 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3fbc4205-d1c7-4938-876b-dc91d7201e6b tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:48.176129 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3fbc4205-d1c7-4938-876b-dc91d7201e6b tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:48.176551 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3fbc4205-d1c7-4938-876b-dc91d7201e6b tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:48.308231 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-3fbc4205-d1c7-4938-876b-dc91d7201e6b tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5975379433a4e31a242764f58954f43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc5ea7a1b6a54cd2af5d648f09aaed63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:48.315631 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-3fbc4205-d1c7-4938-876b-dc91d7201e6b tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5" status: 200 len: 1358 microversion: 2.1 time: 0.152650 Aug 08 02:34:48.316190 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 530/1050] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:48 2022] GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5 => generated 1358 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:48.633506 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-591e94ac-fff5-4f77-92c8-d1b3bae41eac tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:48.634119 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-591e94ac-fff5-4f77-92c8-d1b3bae41eac tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: f0d7098c-da43-4381-99e9-b4756dde20c8] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:48.640001 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-591e94ac-fff5-4f77-92c8-d1b3bae41eac tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:48.640481 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-591e94ac-fff5-4f77-92c8-d1b3bae41eac tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:48.640944 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-591e94ac-fff5-4f77-92c8-d1b3bae41eac tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:48.641625 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-591e94ac-fff5-4f77-92c8-d1b3bae41eac tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:48.642073 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-591e94ac-fff5-4f77-92c8-d1b3bae41eac tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:48.642493 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-591e94ac-fff5-4f77-92c8-d1b3bae41eac tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:48.770404 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-591e94ac-fff5-4f77-92c8-d1b3bae41eac tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:48.777833 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-591e94ac-fff5-4f77-92c8-d1b3bae41eac tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8" status: 200 len: 1267 microversion: 2.1 time: 0.149214 Aug 08 02:34:48.778390 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 521/1051] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:48 2022] GET /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8 => generated 1267 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:49.337690 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-6298fcf1-e881-4985-87a6-500f3dec4052 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:49.338212 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-6298fcf1-e881-4985-87a6-500f3dec4052 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] [instance: 0e099ac0-c453-43b0-8bc1-b4c162d483a5] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:49.344581 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6298fcf1-e881-4985-87a6-500f3dec4052 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:49.345089 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6298fcf1-e881-4985-87a6-500f3dec4052 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:49.345519 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6298fcf1-e881-4985-87a6-500f3dec4052 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:49.346196 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6298fcf1-e881-4985-87a6-500f3dec4052 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:49.346647 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6298fcf1-e881-4985-87a6-500f3dec4052 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:49.347067 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6298fcf1-e881-4985-87a6-500f3dec4052 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:49.486167 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-6298fcf1-e881-4985-87a6-500f3dec4052 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5975379433a4e31a242764f58954f43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc5ea7a1b6a54cd2af5d648f09aaed63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:49.493874 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-6298fcf1-e881-4985-87a6-500f3dec4052 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5" status: 200 len: 1549 microversion: 2.1 time: 0.160599 Aug 08 02:34:49.494445 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 531/1052] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:49 2022] GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5 => generated 1549 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:49.515382 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-6c652d14-0d90-4296-af13-f2c3e3aff5ff tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:49.516001 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-6c652d14-0d90-4296-af13-f2c3e3aff5ff tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] [instance: 0e099ac0-c453-43b0-8bc1-b4c162d483a5] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:49.522129 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6c652d14-0d90-4296-af13-f2c3e3aff5ff tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:49.522612 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6c652d14-0d90-4296-af13-f2c3e3aff5ff tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:49.523686 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6c652d14-0d90-4296-af13-f2c3e3aff5ff tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:49.524476 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6c652d14-0d90-4296-af13-f2c3e3aff5ff tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:49.524967 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6c652d14-0d90-4296-af13-f2c3e3aff5ff tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:49.525395 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6c652d14-0d90-4296-af13-f2c3e3aff5ff tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:49.657021 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-6c652d14-0d90-4296-af13-f2c3e3aff5ff tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5975379433a4e31a242764f58954f43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc5ea7a1b6a54cd2af5d648f09aaed63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:49.664682 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-6c652d14-0d90-4296-af13-f2c3e3aff5ff tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5" status: 200 len: 1549 microversion: 2.1 time: 0.153190 Aug 08 02:34:49.665248 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 522/1053] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:49 2022] GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5 => generated 1549 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:49.684886 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-279e5485-259d-40e6-9a8b-56a537a85256 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:49.689824 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-279e5485-259d-40e6-9a8b-56a537a85256 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'f5975379433a4e31a242764f58954f43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc5ea7a1b6a54cd2af5d648f09aaed63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:49.691654 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-279e5485-259d-40e6-9a8b-56a537a85256 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Searching by: {'deleted': False, 'project_id': 'dc5ea7a1b6a54cd2af5d648f09aaed63'} {{(pid=118352) get_all /opt/stack/nova/nova/compute/api.py:2964}} Aug 08 02:34:49.698741 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-279e5485-259d-40e6-9a8b-56a537a85256 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:49.701726 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-279e5485-259d-40e6-9a8b-56a537a85256 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:49.701726 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-279e5485-259d-40e6-9a8b-56a537a85256 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:49.701726 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-279e5485-259d-40e6-9a8b-56a537a85256 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:49.701726 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-279e5485-259d-40e6-9a8b-56a537a85256 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:49.702116 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-279e5485-259d-40e6-9a8b-56a537a85256 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:49.791279 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.multi_cell_list [None req-279e5485-259d-40e6-9a8b-56a537a85256 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=118352) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Aug 08 02:34:49.799514 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-cc1435c1-7ccb-4551-8165-447bb1ee3d73 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:49.800213 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-cc1435c1-7ccb-4551-8165-447bb1ee3d73 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: f0d7098c-da43-4381-99e9-b4756dde20c8] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:49.807197 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-cc1435c1-7ccb-4551-8165-447bb1ee3d73 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:49.807805 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-cc1435c1-7ccb-4551-8165-447bb1ee3d73 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:49.808305 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-cc1435c1-7ccb-4551-8165-447bb1ee3d73 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:49.809033 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-cc1435c1-7ccb-4551-8165-447bb1ee3d73 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:49.809516 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-cc1435c1-7ccb-4551-8165-447bb1ee3d73 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:49.816324 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-cc1435c1-7ccb-4551-8165-447bb1ee3d73 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:49.888630 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-279e5485-259d-40e6-9a8b-56a537a85256 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "GET /compute/v2.1/servers" status: 200 len: 343 microversion: 2.1 time: 0.207470 Aug 08 02:34:49.889583 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 532/1054] 10.222.0.27 () {58 vars in 1125 bytes} [Mon Aug 8 02:34:49 2022] GET /compute/v2.1/servers => generated 343 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:34:49.900105 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-8a736def-dd29-4778-95ed-c01c91680866 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:49.900779 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-8a736def-dd29-4778-95ed-c01c91680866 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] [instance: 0e099ac0-c453-43b0-8bc1-b4c162d483a5] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:49.906772 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8a736def-dd29-4778-95ed-c01c91680866 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:49.907260 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8a736def-dd29-4778-95ed-c01c91680866 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:49.907699 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8a736def-dd29-4778-95ed-c01c91680866 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:49.908393 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8a736def-dd29-4778-95ed-c01c91680866 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:49.908895 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8a736def-dd29-4778-95ed-c01c91680866 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:49.909326 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8a736def-dd29-4778-95ed-c01c91680866 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:49.950177 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-cc1435c1-7ccb-4551-8165-447bb1ee3d73 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:49.957780 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-cc1435c1-7ccb-4551-8165-447bb1ee3d73 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8" status: 200 len: 1267 microversion: 2.1 time: 0.162676 Aug 08 02:34:49.958355 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 523/1055] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:49 2022] GET /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8 => generated 1267 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:34:50.046047 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-8a736def-dd29-4778-95ed-c01c91680866 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5975379433a4e31a242764f58954f43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc5ea7a1b6a54cd2af5d648f09aaed63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:50.053593 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-8a736def-dd29-4778-95ed-c01c91680866 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5" status: 200 len: 1549 microversion: 2.1 time: 0.157353 Aug 08 02:34:50.054653 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 533/1056] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:49 2022] GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5 => generated 1549 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:34:50.979087 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-2978c7fc-a3fb-4fd4-a14d-2d0105645b0c tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:50.979612 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-2978c7fc-a3fb-4fd4-a14d-2d0105645b0c tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: f0d7098c-da43-4381-99e9-b4756dde20c8] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:50.986066 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2978c7fc-a3fb-4fd4-a14d-2d0105645b0c tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:50.986550 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2978c7fc-a3fb-4fd4-a14d-2d0105645b0c tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:50.986991 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2978c7fc-a3fb-4fd4-a14d-2d0105645b0c tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:50.987682 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2978c7fc-a3fb-4fd4-a14d-2d0105645b0c tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:50.988142 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2978c7fc-a3fb-4fd4-a14d-2d0105645b0c tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:50.988571 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2978c7fc-a3fb-4fd4-a14d-2d0105645b0c tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:51.120441 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-2978c7fc-a3fb-4fd4-a14d-2d0105645b0c tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:51.127956 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-2978c7fc-a3fb-4fd4-a14d-2d0105645b0c tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8" status: 200 len: 1267 microversion: 2.1 time: 0.153578 Aug 08 02:34:51.128525 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 524/1057] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:50 2022] GET /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8 => generated 1267 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:51.899884 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-631871b2-c881-4143-abaa-9eeaa48c76a0 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "f2ca7843-6d4d-4fc9-99e3-60fb4027b403"}} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:34:51.902231 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-631871b2-c881-4143-abaa-9eeaa48c76a0 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] [instance: 0e099ac0-c453-43b0-8bc1-b4c162d483a5] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:51.904648 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell0:SELECT=6 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:51.909751 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-631871b2-c881-4143-abaa-9eeaa48c76a0 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:51.909873 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-631871b2-c881-4143-abaa-9eeaa48c76a0 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:51.910281 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-631871b2-c881-4143-abaa-9eeaa48c76a0 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:51.910995 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-631871b2-c881-4143-abaa-9eeaa48c76a0 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:51.911451 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-631871b2-c881-4143-abaa-9eeaa48c76a0 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:51.911873 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-631871b2-c881-4143-abaa-9eeaa48c76a0 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:52.148539 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-4b77007b-6fc6-42ab-ab3d-a22a3b1491f0 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:52.149108 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-4b77007b-6fc6-42ab-ab3d-a22a3b1491f0 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: f0d7098c-da43-4381-99e9-b4756dde20c8] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:52.151097 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell0:SELECT=2 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:52.156318 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4b77007b-6fc6-42ab-ab3d-a22a3b1491f0 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:52.156840 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4b77007b-6fc6-42ab-ab3d-a22a3b1491f0 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:52.163742 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4b77007b-6fc6-42ab-ab3d-a22a3b1491f0 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:52.163742 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4b77007b-6fc6-42ab-ab3d-a22a3b1491f0 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:52.163742 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4b77007b-6fc6-42ab-ab3d-a22a3b1491f0 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:52.163742 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4b77007b-6fc6-42ab-ab3d-a22a3b1491f0 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:52.293761 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-4b77007b-6fc6-42ab-ab3d-a22a3b1491f0 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:52.301258 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-4b77007b-6fc6-42ab-ab3d-a22a3b1491f0 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8" status: 200 len: 1267 microversion: 2.1 time: 0.156632 Aug 08 02:34:52.301877 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 525/1058] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:52 2022] GET /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8 => generated 1267 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:34:53.057503 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-631871b2-c881-4143-abaa-9eeaa48c76a0 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "POST /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.162258 Aug 08 02:34:53.058127 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 534/1059] 10.222.0.27 () {60 vars in 1323 bytes} [Mon Aug 8 02:34:51 2022] POST /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5/os-volume_attachments => generated 194 bytes in 1164 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:34:53.323080 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-26e28d46-5957-494a-b662-a679a7def589 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:53.323640 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-26e28d46-5957-494a-b662-a679a7def589 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: f0d7098c-da43-4381-99e9-b4756dde20c8] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:53.330461 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-26e28d46-5957-494a-b662-a679a7def589 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:53.331026 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-26e28d46-5957-494a-b662-a679a7def589 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:53.331470 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-26e28d46-5957-494a-b662-a679a7def589 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:53.332167 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-26e28d46-5957-494a-b662-a679a7def589 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:53.332627 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-26e28d46-5957-494a-b662-a679a7def589 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:53.333084 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-26e28d46-5957-494a-b662-a679a7def589 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:53.337961 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell1:SELECT=100 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:53.466367 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-26e28d46-5957-494a-b662-a679a7def589 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:53.473895 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-26e28d46-5957-494a-b662-a679a7def589 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8" status: 200 len: 1267 microversion: 2.1 time: 0.155424 Aug 08 02:34:53.474510 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 526/1060] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:53 2022] GET /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8 => generated 1267 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:54.496607 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-e503dc60-2dca-4c1b-a6d1-dc22351a7fa5 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:54.497334 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-e503dc60-2dca-4c1b-a6d1-dc22351a7fa5 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: f0d7098c-da43-4381-99e9-b4756dde20c8] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:54.503992 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e503dc60-2dca-4c1b-a6d1-dc22351a7fa5 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:54.504469 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e503dc60-2dca-4c1b-a6d1-dc22351a7fa5 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:54.504937 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e503dc60-2dca-4c1b-a6d1-dc22351a7fa5 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:54.505629 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e503dc60-2dca-4c1b-a6d1-dc22351a7fa5 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:54.506083 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e503dc60-2dca-4c1b-a6d1-dc22351a7fa5 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:54.506501 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e503dc60-2dca-4c1b-a6d1-dc22351a7fa5 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:54.638308 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-e503dc60-2dca-4c1b-a6d1-dc22351a7fa5 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:54.645786 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-e503dc60-2dca-4c1b-a6d1-dc22351a7fa5 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8" status: 200 len: 1267 microversion: 2.1 time: 0.154069 Aug 08 02:34:54.646393 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 535/1061] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:54 2022] GET /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8 => generated 1267 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:55.667593 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-4f5a7c8c-5c59-49e1-af88-2acf80fc075f tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:55.668213 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-4f5a7c8c-5c59-49e1-af88-2acf80fc075f tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: f0d7098c-da43-4381-99e9-b4756dde20c8] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:55.674677 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4f5a7c8c-5c59-49e1-af88-2acf80fc075f tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:55.675158 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4f5a7c8c-5c59-49e1-af88-2acf80fc075f tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:55.675592 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4f5a7c8c-5c59-49e1-af88-2acf80fc075f tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:55.676276 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4f5a7c8c-5c59-49e1-af88-2acf80fc075f tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:55.676757 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4f5a7c8c-5c59-49e1-af88-2acf80fc075f tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:55.677183 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-4f5a7c8c-5c59-49e1-af88-2acf80fc075f tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:55.811448 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-4f5a7c8c-5c59-49e1-af88-2acf80fc075f tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:55.819010 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-4f5a7c8c-5c59-49e1-af88-2acf80fc075f tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8" status: 200 len: 1267 microversion: 2.1 time: 0.155851 Aug 08 02:34:55.819569 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 527/1062] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:55 2022] GET /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8 => generated 1267 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:56.552925 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [req-2e9e4bc0-9aba-49b3-af4c-fe5338625afb req-c2609272-9dd5-4936-9c57-82d358d879a4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f0d7098c-da43-4381-99e9-b4756dde20c8", "name": "network-vif-plugged", "status": "completed", "tag": "80747251-b558-4d89-ba58-3fbb7af11ff4"}]} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:34:56.563341 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-2e9e4bc0-9aba-49b3-af4c-fe5338625afb req-c2609272-9dd5-4936-9c57-82d358d879a4 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:56.563820 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-2e9e4bc0-9aba-49b3-af4c-fe5338625afb req-c2609272-9dd5-4936-9c57-82d358d879a4 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:56.564251 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-2e9e4bc0-9aba-49b3-af4c-fe5338625afb req-c2609272-9dd5-4936-9c57-82d358d879a4 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:56.568018 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:56.575977 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.compute.server_external_events [req-2e9e4bc0-9aba-49b3-af4c-fe5338625afb req-c2609272-9dd5-4936-9c57-82d358d879a4 service nova] Creating event network-vif-plugged:80747251-b558-4d89-ba58-3fbb7af11ff4 for instance f0d7098c-da43-4381-99e9-b4756dde20c8 on hv-cinder-74014 Aug 08 02:34:56.580719 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [req-2e9e4bc0-9aba-49b3-af4c-fe5338625afb req-c2609272-9dd5-4936-9c57-82d358d879a4 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031475 Aug 08 02:34:56.581303 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 536/1063] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:34:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:34:56.839602 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-58b31cac-82cb-4ab6-8cc9-993475e6efa5 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:56.840159 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-58b31cac-82cb-4ab6-8cc9-993475e6efa5 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: f0d7098c-da43-4381-99e9-b4756dde20c8] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:56.846817 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-58b31cac-82cb-4ab6-8cc9-993475e6efa5 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:56.847292 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-58b31cac-82cb-4ab6-8cc9-993475e6efa5 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:56.847728 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-58b31cac-82cb-4ab6-8cc9-993475e6efa5 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:56.848408 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-58b31cac-82cb-4ab6-8cc9-993475e6efa5 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:56.848885 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-58b31cac-82cb-4ab6-8cc9-993475e6efa5 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:56.849307 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-58b31cac-82cb-4ab6-8cc9-993475e6efa5 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:56.984405 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-58b31cac-82cb-4ab6-8cc9-993475e6efa5 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:56.993442 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-58b31cac-82cb-4ab6-8cc9-993475e6efa5 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8" status: 200 len: 1267 microversion: 2.1 time: 0.157696 Aug 08 02:34:56.994023 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 528/1064] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:56 2022] GET /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8 => generated 1267 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:58.016193 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-292532ea-94a7-48da-8f38-c2959cb86e00 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:58.016926 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-292532ea-94a7-48da-8f38-c2959cb86e00 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: f0d7098c-da43-4381-99e9-b4756dde20c8] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:58.024124 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-292532ea-94a7-48da-8f38-c2959cb86e00 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:58.024610 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-292532ea-94a7-48da-8f38-c2959cb86e00 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:58.028731 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-292532ea-94a7-48da-8f38-c2959cb86e00 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:58.028731 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-292532ea-94a7-48da-8f38-c2959cb86e00 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:58.028731 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-292532ea-94a7-48da-8f38-c2959cb86e00 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:58.028731 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-292532ea-94a7-48da-8f38-c2959cb86e00 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:58.162089 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-292532ea-94a7-48da-8f38-c2959cb86e00 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:58.170492 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-292532ea-94a7-48da-8f38-c2959cb86e00 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8" status: 200 len: 1267 microversion: 2.1 time: 0.159412 Aug 08 02:34:58.171051 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 537/1065] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:58 2022] GET /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8 => generated 1267 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:59.191790 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-6f293f4f-d715-40c9-80fb-feb4b54052d9 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:59.192334 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-6f293f4f-d715-40c9-80fb-feb4b54052d9 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: f0d7098c-da43-4381-99e9-b4756dde20c8] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:59.199357 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6f293f4f-d715-40c9-80fb-feb4b54052d9 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:59.199837 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6f293f4f-d715-40c9-80fb-feb4b54052d9 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:59.200270 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6f293f4f-d715-40c9-80fb-feb4b54052d9 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:59.200985 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6f293f4f-d715-40c9-80fb-feb4b54052d9 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:59.201437 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6f293f4f-d715-40c9-80fb-feb4b54052d9 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:59.201856 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6f293f4f-d715-40c9-80fb-feb4b54052d9 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:59.336565 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-6f293f4f-d715-40c9-80fb-feb4b54052d9 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:59.344237 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-6f293f4f-d715-40c9-80fb-feb4b54052d9 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8" status: 200 len: 1454 microversion: 2.1 time: 0.156420 Aug 08 02:34:59.344831 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 529/1066] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:59 2022] GET /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8 => generated 1454 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:34:59.364713 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-aa9dcfe6-71e5-42b7-bde7-bcb28c7537e7 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:34:59.365351 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-aa9dcfe6-71e5-42b7-bde7-bcb28c7537e7 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: f0d7098c-da43-4381-99e9-b4756dde20c8] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:34:59.372920 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-aa9dcfe6-71e5-42b7-bde7-bcb28c7537e7 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:59.374300 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-aa9dcfe6-71e5-42b7-bde7-bcb28c7537e7 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:59.374300 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-aa9dcfe6-71e5-42b7-bde7-bcb28c7537e7 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:59.374571 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-aa9dcfe6-71e5-42b7-bde7-bcb28c7537e7 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:34:59.374998 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-aa9dcfe6-71e5-42b7-bde7-bcb28c7537e7 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:34:59.375424 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-aa9dcfe6-71e5-42b7-bde7-bcb28c7537e7 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:34:59.509957 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-aa9dcfe6-71e5-42b7-bde7-bcb28c7537e7 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:34:59.517434 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-aa9dcfe6-71e5-42b7-bde7-bcb28c7537e7 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8" status: 200 len: 1454 microversion: 2.1 time: 0.156579 Aug 08 02:34:59.517995 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 538/1067] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:34:59 2022] GET /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8 => generated 1454 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:35:00.642830 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-7ae58bba-d016-4cfa-9559-390c6e64b317 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:35:00.643450 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-7ae58bba-d016-4cfa-9559-390c6e64b317 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] [instance: 0e099ac0-c453-43b0-8bc1-b4c162d483a5] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:35:00.650215 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7ae58bba-d016-4cfa-9559-390c6e64b317 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:35:00.650703 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7ae58bba-d016-4cfa-9559-390c6e64b317 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:35:00.651142 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7ae58bba-d016-4cfa-9559-390c6e64b317 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:35:00.651839 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7ae58bba-d016-4cfa-9559-390c6e64b317 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:35:00.652294 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7ae58bba-d016-4cfa-9559-390c6e64b317 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:35:00.652748 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7ae58bba-d016-4cfa-9559-390c6e64b317 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:35:00.783542 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-7ae58bba-d016-4cfa-9559-390c6e64b317 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5975379433a4e31a242764f58954f43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc5ea7a1b6a54cd2af5d648f09aaed63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:35:00.792200 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-7ae58bba-d016-4cfa-9559-390c6e64b317 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5" status: 200 len: 1595 microversion: 2.1 time: 0.153411 Aug 08 02:35:00.792791 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 530/1068] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:35:00 2022] GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5 => generated 1595 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:35:01.594576 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [req-1e2dea3c-20dc-4cfd-97f9-ee1e14325e3c req-cae0e104-9e28-40e9-bc50-f790803568fb service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f0d7098c-da43-4381-99e9-b4756dde20c8", "tag": "80747251-b558-4d89-ba58-3fbb7af11ff4"}]} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:35:01.599381 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell0:SELECT=4 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:01.602113 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-1e2dea3c-20dc-4cfd-97f9-ee1e14325e3c req-cae0e104-9e28-40e9-bc50-f790803568fb service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:35:01.602814 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-1e2dea3c-20dc-4cfd-97f9-ee1e14325e3c req-cae0e104-9e28-40e9-bc50-f790803568fb service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:35:01.603575 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-1e2dea3c-20dc-4cfd-97f9-ee1e14325e3c req-cae0e104-9e28-40e9-bc50-f790803568fb service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:35:01.615335 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.compute.server_external_events [req-1e2dea3c-20dc-4cfd-97f9-ee1e14325e3c req-cae0e104-9e28-40e9-bc50-f790803568fb service nova] Creating event network-changed:80747251-b558-4d89-ba58-3fbb7af11ff4 for instance f0d7098c-da43-4381-99e9-b4756dde20c8 on hv-cinder-74014 Aug 08 02:35:01.620885 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [req-1e2dea3c-20dc-4cfd-97f9-ee1e14325e3c req-cae0e104-9e28-40e9-bc50-f790803568fb service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039955 Aug 08 02:35:01.621868 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 539/1069] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:35:01 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:35:03.080276 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [req-6584242c-d58b-4e39-947c-45680d899080 req-5c4c3533-1d38-4311-b6b4-da6de76a63c6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0e099ac0-c453-43b0-8bc1-b4c162d483a5", "tag": "35cd469b-968e-46d3-8566-8026ce405507"}]} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:35:03.091666 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-6584242c-d58b-4e39-947c-45680d899080 req-5c4c3533-1d38-4311-b6b4-da6de76a63c6 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:35:03.092505 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-6584242c-d58b-4e39-947c-45680d899080 req-5c4c3533-1d38-4311-b6b4-da6de76a63c6 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:35:03.093205 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-6584242c-d58b-4e39-947c-45680d899080 req-5c4c3533-1d38-4311-b6b4-da6de76a63c6 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:35:03.105555 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.compute.server_external_events [req-6584242c-d58b-4e39-947c-45680d899080 req-5c4c3533-1d38-4311-b6b4-da6de76a63c6 service nova] Creating event network-changed:35cd469b-968e-46d3-8566-8026ce405507 for instance 0e099ac0-c453-43b0-8bc1-b4c162d483a5 on hv-cinder-74014 Aug 08 02:35:03.110693 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [req-6584242c-d58b-4e39-947c-45680d899080 req-5c4c3533-1d38-4311-b6b4-da6de76a63c6 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034471 Aug 08 02:35:03.111280 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 531/1070] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:35:03 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 08 02:35:05.904581 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-6e6cc32b-f4a5-4898-a800-6b3c348a11c2 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-secgroup-smoke-925982175"}} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:35:05.905210 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-6e6cc32b-f4a5-4898-a800-6b3c348a11c2 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] [instance: 0e099ac0-c453-43b0-8bc1-b4c162d483a5] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:35:05.911777 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6e6cc32b-f4a5-4898-a800-6b3c348a11c2 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:35:05.912254 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6e6cc32b-f4a5-4898-a800-6b3c348a11c2 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:35:05.912707 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6e6cc32b-f4a5-4898-a800-6b3c348a11c2 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:35:05.914019 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6e6cc32b-f4a5-4898-a800-6b3c348a11c2 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:35:05.914465 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6e6cc32b-f4a5-4898-a800-6b3c348a11c2 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:35:05.914916 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6e6cc32b-f4a5-4898-a800-6b3c348a11c2 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:35:06.045174 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.network.security_group_api [None req-6e6cc32b-f4a5-4898-a800-6b3c348a11c2 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Adding security group 52ece7e3-75cf-4d6d-97d5-3b3ecc4c8c3f to port 35cd469b-968e-46d3-8566-8026ce405507 Aug 08 02:35:06.447562 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-6e6cc32b-f4a5-4898-a800-6b3c348a11c2 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "POST /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5/action" status: 202 len: 0 microversion: 2.1 time: 0.547366 Aug 08 02:35:06.448567 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 540/1071] 10.222.0.27 () {60 vars in 1278 bytes} [Mon Aug 8 02:35:05 2022] POST /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5/action => generated 0 bytes in 550 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 08 02:35:06.458357 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-3c5cac2e-9542-4c3a-8a69-0611861102ef tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:35:06.459675 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-3c5cac2e-9542-4c3a-8a69-0611861102ef tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] [instance: 0e099ac0-c453-43b0-8bc1-b4c162d483a5] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:35:06.462320 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_api:SELECT=48 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:06.467512 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3c5cac2e-9542-4c3a-8a69-0611861102ef tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:35:06.468013 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3c5cac2e-9542-4c3a-8a69-0611861102ef tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:35:06.468450 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3c5cac2e-9542-4c3a-8a69-0611861102ef tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:35:06.469212 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3c5cac2e-9542-4c3a-8a69-0611861102ef tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:35:06.470208 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3c5cac2e-9542-4c3a-8a69-0611861102ef tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:35:06.470208 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3c5cac2e-9542-4c3a-8a69-0611861102ef tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:35:06.493755 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [req-8953453d-8a97-4fdf-8f73-14d448fd6dbf req-ec6de063-7b2e-477a-8342-1742b6f95a17 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0e099ac0-c453-43b0-8bc1-b4c162d483a5", "tag": "35cd469b-968e-46d3-8566-8026ce405507"}]} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:35:06.505090 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_api:SELECT=57,nova_api:INSERT=3 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:06.516524 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-8953453d-8a97-4fdf-8f73-14d448fd6dbf req-ec6de063-7b2e-477a-8342-1742b6f95a17 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:35:06.517216 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-8953453d-8a97-4fdf-8f73-14d448fd6dbf req-ec6de063-7b2e-477a-8342-1742b6f95a17 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:35:06.517937 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-8953453d-8a97-4fdf-8f73-14d448fd6dbf req-ec6de063-7b2e-477a-8342-1742b6f95a17 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:35:06.531291 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.compute.server_external_events [req-8953453d-8a97-4fdf-8f73-14d448fd6dbf req-ec6de063-7b2e-477a-8342-1742b6f95a17 service nova] Creating event network-changed:35cd469b-968e-46d3-8566-8026ce405507 for instance 0e099ac0-c453-43b0-8bc1-b4c162d483a5 on hv-cinder-74014 Aug 08 02:35:06.536532 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [req-8953453d-8a97-4fdf-8f73-14d448fd6dbf req-ec6de063-7b2e-477a-8342-1742b6f95a17 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.052461 Aug 08 02:35:06.537459 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 541/1072] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:35:06 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:35:06.675735 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-3c5cac2e-9542-4c3a-8a69-0611861102ef tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5975379433a4e31a242764f58954f43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc5ea7a1b6a54cd2af5d648f09aaed63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:35:06.686271 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-3c5cac2e-9542-4c3a-8a69-0611861102ef tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5" status: 200 len: 1760 microversion: 2.1 time: 0.232655 Aug 08 02:35:06.689040 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 532/1073] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:35:06 2022] GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5 => generated 1760 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:35:23.863079 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-bab7a0d9-d1ef-4415-bf58-03c6936e7bd5 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:35:23.865318 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-bab7a0d9-d1ef-4415-bf58-03c6936e7bd5 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] [instance: 0e099ac0-c453-43b0-8bc1-b4c162d483a5] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:35:23.868116 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell1:SELECT=21 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:23.874077 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-bab7a0d9-d1ef-4415-bf58-03c6936e7bd5 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:35:23.874591 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-bab7a0d9-d1ef-4415-bf58-03c6936e7bd5 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:35:23.875065 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-bab7a0d9-d1ef-4415-bf58-03c6936e7bd5 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:35:23.875746 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-bab7a0d9-d1ef-4415-bf58-03c6936e7bd5 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:35:23.876197 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-bab7a0d9-d1ef-4415-bf58-03c6936e7bd5 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:35:23.876618 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-bab7a0d9-d1ef-4415-bf58-03c6936e7bd5 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:35:24.206570 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-bab7a0d9-d1ef-4415-bf58-03c6936e7bd5 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "POST /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5/action" status: 202 len: 0 microversion: 2.1 time: 0.347660 Aug 08 02:35:24.207163 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 542/1074] 10.222.0.27 () {60 vars in 1278 bytes} [Mon Aug 8 02:35:23 2022] POST /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5/action => generated 0 bytes in 349 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 08 02:35:24.220242 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-f68e8a82-deb1-49ff-b636-7191fa7db5d8 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:35:24.220958 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-f68e8a82-deb1-49ff-b636-7191fa7db5d8 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] [instance: 0e099ac0-c453-43b0-8bc1-b4c162d483a5] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:35:24.223181 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_api:SELECT=1 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:24.223824 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell1:SELECT=36 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:24.228511 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f68e8a82-deb1-49ff-b636-7191fa7db5d8 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:35:24.230477 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f68e8a82-deb1-49ff-b636-7191fa7db5d8 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:35:24.230735 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f68e8a82-deb1-49ff-b636-7191fa7db5d8 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:35:24.231428 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f68e8a82-deb1-49ff-b636-7191fa7db5d8 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:35:24.231878 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f68e8a82-deb1-49ff-b636-7191fa7db5d8 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:35:24.232301 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f68e8a82-deb1-49ff-b636-7191fa7db5d8 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:35:24.488600 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-f68e8a82-deb1-49ff-b636-7191fa7db5d8 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5975379433a4e31a242764f58954f43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc5ea7a1b6a54cd2af5d648f09aaed63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:35:24.497277 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-f68e8a82-deb1-49ff-b636-7191fa7db5d8 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5" status: 200 len: 1752 microversion: 2.1 time: 0.280972 Aug 08 02:35:24.497896 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 533/1075] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:35:24 2022] GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5 => generated 1752 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:35:25.519587 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-b65c2e1e-eda5-4c24-bd6e-e7b9e880ae58 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:35:25.520287 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-b65c2e1e-eda5-4c24-bd6e-e7b9e880ae58 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] [instance: 0e099ac0-c453-43b0-8bc1-b4c162d483a5] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:35:25.526547 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b65c2e1e-eda5-4c24-bd6e-e7b9e880ae58 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:35:25.527025 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b65c2e1e-eda5-4c24-bd6e-e7b9e880ae58 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:35:25.527461 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b65c2e1e-eda5-4c24-bd6e-e7b9e880ae58 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:35:25.528141 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b65c2e1e-eda5-4c24-bd6e-e7b9e880ae58 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:35:25.528596 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b65c2e1e-eda5-4c24-bd6e-e7b9e880ae58 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:35:25.529046 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b65c2e1e-eda5-4c24-bd6e-e7b9e880ae58 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:35:25.726380 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-b65c2e1e-eda5-4c24-bd6e-e7b9e880ae58 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5975379433a4e31a242764f58954f43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc5ea7a1b6a54cd2af5d648f09aaed63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:35:25.735644 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-b65c2e1e-eda5-4c24-bd6e-e7b9e880ae58 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5" status: 200 len: 1757 microversion: 2.1 time: 0.220638 Aug 08 02:35:25.736746 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 543/1076] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:35:25 2022] GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5 => generated 1757 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:35:26.758374 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-00fd37b4-b974-4e17-81bd-137a82868935 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:35:26.759391 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-00fd37b4-b974-4e17-81bd-137a82868935 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] [instance: 0e099ac0-c453-43b0-8bc1-b4c162d483a5] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:35:26.766216 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-00fd37b4-b974-4e17-81bd-137a82868935 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:35:26.766704 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-00fd37b4-b974-4e17-81bd-137a82868935 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:35:26.767140 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-00fd37b4-b974-4e17-81bd-137a82868935 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:35:26.767836 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-00fd37b4-b974-4e17-81bd-137a82868935 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:35:26.768290 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-00fd37b4-b974-4e17-81bd-137a82868935 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:35:26.768741 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-00fd37b4-b974-4e17-81bd-137a82868935 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:35:26.915457 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-00fd37b4-b974-4e17-81bd-137a82868935 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5975379433a4e31a242764f58954f43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc5ea7a1b6a54cd2af5d648f09aaed63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:35:26.923980 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-00fd37b4-b974-4e17-81bd-137a82868935 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5" status: 200 len: 1757 microversion: 2.1 time: 0.169675 Aug 08 02:35:26.936997 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 534/1077] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:35:26 2022] GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5 => generated 1757 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:35:27.946183 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-93943074-ad23-485e-8728-49af33178822 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:35:27.946761 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-93943074-ad23-485e-8728-49af33178822 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] [instance: 0e099ac0-c453-43b0-8bc1-b4c162d483a5] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:35:27.954282 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-93943074-ad23-485e-8728-49af33178822 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:35:27.954760 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-93943074-ad23-485e-8728-49af33178822 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:35:27.955191 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-93943074-ad23-485e-8728-49af33178822 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:35:27.955870 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-93943074-ad23-485e-8728-49af33178822 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:35:27.956319 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-93943074-ad23-485e-8728-49af33178822 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:35:27.956772 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-93943074-ad23-485e-8728-49af33178822 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:35:28.097682 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-93943074-ad23-485e-8728-49af33178822 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5975379433a4e31a242764f58954f43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc5ea7a1b6a54cd2af5d648f09aaed63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:35:28.106167 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-93943074-ad23-485e-8728-49af33178822 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5" status: 200 len: 1757 microversion: 2.1 time: 0.163441 Aug 08 02:35:28.106737 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 544/1078] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:35:27 2022] GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5 => generated 1757 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:35:29.127510 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-3fcbab45-c90c-4fa5-95bd-ab53103eca88 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:35:29.128119 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-3fcbab45-c90c-4fa5-95bd-ab53103eca88 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] [instance: 0e099ac0-c453-43b0-8bc1-b4c162d483a5] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:35:29.134568 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3fcbab45-c90c-4fa5-95bd-ab53103eca88 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:35:29.135049 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3fcbab45-c90c-4fa5-95bd-ab53103eca88 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:35:29.135483 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3fcbab45-c90c-4fa5-95bd-ab53103eca88 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:35:29.136170 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3fcbab45-c90c-4fa5-95bd-ab53103eca88 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:35:29.136624 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3fcbab45-c90c-4fa5-95bd-ab53103eca88 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:35:29.137076 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3fcbab45-c90c-4fa5-95bd-ab53103eca88 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:35:29.282002 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-3fcbab45-c90c-4fa5-95bd-ab53103eca88 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5975379433a4e31a242764f58954f43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc5ea7a1b6a54cd2af5d648f09aaed63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:35:29.289957 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-3fcbab45-c90c-4fa5-95bd-ab53103eca88 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5" status: 200 len: 1757 microversion: 2.1 time: 0.166359 Aug 08 02:35:29.290524 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 535/1079] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:35:29 2022] GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5 => generated 1757 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:35:30.313394 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-e094f6bc-6c41-43d2-a6f0-e5d474330976 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:35:30.313982 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-e094f6bc-6c41-43d2-a6f0-e5d474330976 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] [instance: 0e099ac0-c453-43b0-8bc1-b4c162d483a5] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:35:30.319977 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e094f6bc-6c41-43d2-a6f0-e5d474330976 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:35:30.320457 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e094f6bc-6c41-43d2-a6f0-e5d474330976 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:35:30.320929 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e094f6bc-6c41-43d2-a6f0-e5d474330976 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:35:30.321613 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e094f6bc-6c41-43d2-a6f0-e5d474330976 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:35:30.322065 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e094f6bc-6c41-43d2-a6f0-e5d474330976 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:35:30.322487 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e094f6bc-6c41-43d2-a6f0-e5d474330976 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:35:30.464985 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-e094f6bc-6c41-43d2-a6f0-e5d474330976 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5975379433a4e31a242764f58954f43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc5ea7a1b6a54cd2af5d648f09aaed63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:35:30.473569 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-e094f6bc-6c41-43d2-a6f0-e5d474330976 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5" status: 200 len: 1757 microversion: 2.1 time: 0.165006 Aug 08 02:35:30.474134 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 545/1080] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:35:30 2022] GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5 => generated 1757 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:35:31.495298 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-99c12aec-f3d5-47de-8740-dbd5fdbe7dd4 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:35:31.495944 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-99c12aec-f3d5-47de-8740-dbd5fdbe7dd4 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] [instance: 0e099ac0-c453-43b0-8bc1-b4c162d483a5] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:35:31.502646 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-99c12aec-f3d5-47de-8740-dbd5fdbe7dd4 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:35:31.503136 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-99c12aec-f3d5-47de-8740-dbd5fdbe7dd4 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:35:31.503637 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-99c12aec-f3d5-47de-8740-dbd5fdbe7dd4 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:35:31.504329 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-99c12aec-f3d5-47de-8740-dbd5fdbe7dd4 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:35:31.504884 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-99c12aec-f3d5-47de-8740-dbd5fdbe7dd4 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:35:31.505312 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-99c12aec-f3d5-47de-8740-dbd5fdbe7dd4 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:35:31.650376 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-99c12aec-f3d5-47de-8740-dbd5fdbe7dd4 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5975379433a4e31a242764f58954f43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc5ea7a1b6a54cd2af5d648f09aaed63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:35:31.658510 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-99c12aec-f3d5-47de-8740-dbd5fdbe7dd4 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5" status: 200 len: 1757 microversion: 2.1 time: 0.167230 Aug 08 02:35:31.659138 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 536/1081] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:35:31 2022] GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5 => generated 1757 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:35:32.678783 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-1599405f-c84c-4474-8850-680c3ae59bbb tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:35:32.679390 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-1599405f-c84c-4474-8850-680c3ae59bbb tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] [instance: 0e099ac0-c453-43b0-8bc1-b4c162d483a5] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:35:32.686096 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1599405f-c84c-4474-8850-680c3ae59bbb tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:35:32.686575 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1599405f-c84c-4474-8850-680c3ae59bbb tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:35:32.687007 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1599405f-c84c-4474-8850-680c3ae59bbb tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:35:32.687689 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1599405f-c84c-4474-8850-680c3ae59bbb tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:35:32.688139 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1599405f-c84c-4474-8850-680c3ae59bbb tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:35:32.688563 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-1599405f-c84c-4474-8850-680c3ae59bbb tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:35:32.836089 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-1599405f-c84c-4474-8850-680c3ae59bbb tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5975379433a4e31a242764f58954f43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc5ea7a1b6a54cd2af5d648f09aaed63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:35:32.844764 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-1599405f-c84c-4474-8850-680c3ae59bbb tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5" status: 200 len: 1757 microversion: 2.1 time: 0.169705 Aug 08 02:35:32.845268 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 546/1082] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:35:32 2022] GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5 => generated 1757 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:35:33.865112 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-5d4451dd-2a2a-401a-9c24-eff85bb8d4fc tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:35:33.865762 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-5d4451dd-2a2a-401a-9c24-eff85bb8d4fc tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] [instance: 0e099ac0-c453-43b0-8bc1-b4c162d483a5] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:35:33.873496 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5d4451dd-2a2a-401a-9c24-eff85bb8d4fc tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:35:33.874072 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5d4451dd-2a2a-401a-9c24-eff85bb8d4fc tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:35:33.874512 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5d4451dd-2a2a-401a-9c24-eff85bb8d4fc tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:35:33.875279 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5d4451dd-2a2a-401a-9c24-eff85bb8d4fc tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:35:33.875771 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5d4451dd-2a2a-401a-9c24-eff85bb8d4fc tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:35:33.876193 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-5d4451dd-2a2a-401a-9c24-eff85bb8d4fc tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:35:34.021360 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-5d4451dd-2a2a-401a-9c24-eff85bb8d4fc tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5975379433a4e31a242764f58954f43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc5ea7a1b6a54cd2af5d648f09aaed63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:35:34.029759 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-5d4451dd-2a2a-401a-9c24-eff85bb8d4fc tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5" status: 200 len: 1757 microversion: 2.1 time: 0.168500 Aug 08 02:35:34.030334 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 537/1083] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:35:33 2022] GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5 => generated 1757 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:35:35.050917 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-98078dfe-cdd3-444e-a068-0e67c19010f1 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:35:35.051522 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-98078dfe-cdd3-444e-a068-0e67c19010f1 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] [instance: 0e099ac0-c453-43b0-8bc1-b4c162d483a5] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:35:35.058058 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-98078dfe-cdd3-444e-a068-0e67c19010f1 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:35:35.058542 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-98078dfe-cdd3-444e-a068-0e67c19010f1 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:35:35.059004 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-98078dfe-cdd3-444e-a068-0e67c19010f1 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:35:35.060330 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-98078dfe-cdd3-444e-a068-0e67c19010f1 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:35:35.060783 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-98078dfe-cdd3-444e-a068-0e67c19010f1 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:35:35.061219 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-98078dfe-cdd3-444e-a068-0e67c19010f1 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:35:35.202972 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-98078dfe-cdd3-444e-a068-0e67c19010f1 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5975379433a4e31a242764f58954f43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc5ea7a1b6a54cd2af5d648f09aaed63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:35:35.210896 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-98078dfe-cdd3-444e-a068-0e67c19010f1 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5" status: 200 len: 1757 microversion: 2.1 time: 0.163780 Aug 08 02:35:35.212938 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 547/1084] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:35:35 2022] GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5 => generated 1757 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:35:36.232576 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-eb0c1797-94b4-4379-bf21-6c9de9db631c tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:35:36.233257 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-eb0c1797-94b4-4379-bf21-6c9de9db631c tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] [instance: 0e099ac0-c453-43b0-8bc1-b4c162d483a5] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:35:36.239784 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-eb0c1797-94b4-4379-bf21-6c9de9db631c tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:35:36.240268 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-eb0c1797-94b4-4379-bf21-6c9de9db631c tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:35:36.240781 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-eb0c1797-94b4-4379-bf21-6c9de9db631c tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:35:36.241509 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-eb0c1797-94b4-4379-bf21-6c9de9db631c tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:35:36.242008 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-eb0c1797-94b4-4379-bf21-6c9de9db631c tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:35:36.242459 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-eb0c1797-94b4-4379-bf21-6c9de9db631c tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:35:36.385234 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-eb0c1797-94b4-4379-bf21-6c9de9db631c tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5975379433a4e31a242764f58954f43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc5ea7a1b6a54cd2af5d648f09aaed63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:35:36.393304 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-eb0c1797-94b4-4379-bf21-6c9de9db631c tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5" status: 200 len: 1760 microversion: 2.1 time: 0.165297 Aug 08 02:35:36.393871 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 538/1085] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:35:36 2022] GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5 => generated 1760 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:36:09.386286 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-11f6c143-0e6f-4238-b99f-192fb71e5100 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-secgroup-smoke-925982175"}} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:36:09.386898 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-11f6c143-0e6f-4238-b99f-192fb71e5100 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] [instance: 0e099ac0-c453-43b0-8bc1-b4c162d483a5] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:36:09.388939 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_api:SELECT=13 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:09.389738 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell1:SELECT=40,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:09.394337 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-11f6c143-0e6f-4238-b99f-192fb71e5100 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:36:09.394823 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-11f6c143-0e6f-4238-b99f-192fb71e5100 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:36:09.395256 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-11f6c143-0e6f-4238-b99f-192fb71e5100 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:36:09.396554 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-11f6c143-0e6f-4238-b99f-192fb71e5100 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:36:09.397049 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-11f6c143-0e6f-4238-b99f-192fb71e5100 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:36:09.397472 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-11f6c143-0e6f-4238-b99f-192fb71e5100 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:36:09.527870 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.network.security_group_api [None req-11f6c143-0e6f-4238-b99f-192fb71e5100 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Removing security group 52ece7e3-75cf-4d6d-97d5-3b3ecc4c8c3f from port 35cd469b-968e-46d3-8566-8026ce405507 Aug 08 02:36:09.875965 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-11f6c143-0e6f-4238-b99f-192fb71e5100 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "POST /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5/action" status: 202 len: 0 microversion: 2.1 time: 0.490989 Aug 08 02:36:09.875965 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 548/1086] 10.222.0.27 () {60 vars in 1278 bytes} [Mon Aug 8 02:36:09 2022] POST /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5/action => generated 0 bytes in 492 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 08 02:36:09.878644 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_api:SELECT=12 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:09.879447 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell1:SELECT=36 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:09.884428 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [req-20e99a3a-81f4-4b19-a7cf-14e25443a172 req-42a0627a-876c-4431-a928-35e365a19404 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0e099ac0-c453-43b0-8bc1-b4c162d483a5", "tag": "35cd469b-968e-46d3-8566-8026ce405507"}]} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:36:09.893939 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-20e99a3a-81f4-4b19-a7cf-14e25443a172 req-42a0627a-876c-4431-a928-35e365a19404 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:36:09.894772 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-20e99a3a-81f4-4b19-a7cf-14e25443a172 req-42a0627a-876c-4431-a928-35e365a19404 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:36:09.894906 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-20e99a3a-81f4-4b19-a7cf-14e25443a172 req-42a0627a-876c-4431-a928-35e365a19404 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:36:09.906769 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.compute.server_external_events [req-20e99a3a-81f4-4b19-a7cf-14e25443a172 req-42a0627a-876c-4431-a928-35e365a19404 service nova] Creating event network-changed:35cd469b-968e-46d3-8566-8026ce405507 for instance 0e099ac0-c453-43b0-8bc1-b4c162d483a5 on hv-cinder-74014 Aug 08 02:36:09.912241 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [req-20e99a3a-81f4-4b19-a7cf-14e25443a172 req-42a0627a-876c-4431-a928-35e365a19404 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034448 Aug 08 02:36:09.913234 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 539/1087] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:36:09 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:36:11.334598 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-cc776d9c-2469-4d02-92c8-77331105fcfc tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:36:11.335131 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-cc776d9c-2469-4d02-92c8-77331105fcfc tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] [instance: 0e099ac0-c453-43b0-8bc1-b4c162d483a5] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:36:11.341128 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cc776d9c-2469-4d02-92c8-77331105fcfc tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:36:11.341607 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cc776d9c-2469-4d02-92c8-77331105fcfc tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:36:11.342049 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cc776d9c-2469-4d02-92c8-77331105fcfc tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:36:11.342733 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cc776d9c-2469-4d02-92c8-77331105fcfc tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:36:11.343186 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cc776d9c-2469-4d02-92c8-77331105fcfc tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:36:11.343611 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-cc776d9c-2469-4d02-92c8-77331105fcfc tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:36:11.422575 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.objects.instance [None req-cc776d9c-2469-4d02-92c8-77331105fcfc tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lazy-loading 'services' on Instance uuid 0e099ac0-c453-43b0-8bc1-b4c162d483a5 {{(pid=118352) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 08 02:36:11.629900 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-cc776d9c-2469-4d02-92c8-77331105fcfc tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "DELETE /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5/os-volume_attachments/f2ca7843-6d4d-4fc9-99e3-60fb4027b403" status: 202 len: 0 microversion: 2.1 time: 0.300351 Aug 08 02:36:11.631152 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 549/1088] 10.222.0.27 () {58 vars in 1416 bytes} [Mon Aug 8 02:36:11 2022] DELETE /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5/os-volume_attachments/f2ca7843-6d4d-4fc9-99e3-60fb4027b403 => generated 0 bytes in 303 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 08 02:36:11.641434 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-537bde61-9cda-4ba6-a169-81f239c01474 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:36:11.643813 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-537bde61-9cda-4ba6-a169-81f239c01474 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] [instance: 0e099ac0-c453-43b0-8bc1-b4c162d483a5] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:36:11.651288 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-537bde61-9cda-4ba6-a169-81f239c01474 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:36:11.651853 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-537bde61-9cda-4ba6-a169-81f239c01474 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:36:11.652362 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-537bde61-9cda-4ba6-a169-81f239c01474 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:36:11.653155 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-537bde61-9cda-4ba6-a169-81f239c01474 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:36:11.653671 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-537bde61-9cda-4ba6-a169-81f239c01474 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:36:11.654188 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-537bde61-9cda-4ba6-a169-81f239c01474 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:36:11.684799 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-537bde61-9cda-4ba6-a169-81f239c01474 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048863 Aug 08 02:36:11.685366 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 540/1089] 10.222.0.27 () {58 vars in 1302 bytes} [Mon Aug 8 02:36:11 2022] GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:36:11.934440 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [req-bf52e3c3-407e-4e33-a78b-6266f83d6346 req-3abda4de-6381-44ba-a242-5a5ef68ad4f7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0e099ac0-c453-43b0-8bc1-b4c162d483a5", "tag": "35cd469b-968e-46d3-8566-8026ce405507"}]} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:36:11.944363 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-bf52e3c3-407e-4e33-a78b-6266f83d6346 req-3abda4de-6381-44ba-a242-5a5ef68ad4f7 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:36:11.944865 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-bf52e3c3-407e-4e33-a78b-6266f83d6346 req-3abda4de-6381-44ba-a242-5a5ef68ad4f7 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:36:11.945293 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-bf52e3c3-407e-4e33-a78b-6266f83d6346 req-3abda4de-6381-44ba-a242-5a5ef68ad4f7 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:36:11.956518 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.compute.server_external_events [req-bf52e3c3-407e-4e33-a78b-6266f83d6346 req-3abda4de-6381-44ba-a242-5a5ef68ad4f7 service nova] Creating event network-changed:35cd469b-968e-46d3-8566-8026ce405507 for instance 0e099ac0-c453-43b0-8bc1-b4c162d483a5 on hv-cinder-74014 Aug 08 02:36:11.961430 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [req-bf52e3c3-407e-4e33-a78b-6266f83d6346 req-3abda4de-6381-44ba-a242-5a5ef68ad4f7 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030926 Aug 08 02:36:11.962434 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 550/1090] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:36:11 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:36:12.697883 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-f1ca63a9-1aaa-4f90-99b8-302a6ff5449c tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:36:12.700693 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-f1ca63a9-1aaa-4f90-99b8-302a6ff5449c tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] [instance: 0e099ac0-c453-43b0-8bc1-b4c162d483a5] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:36:12.709199 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f1ca63a9-1aaa-4f90-99b8-302a6ff5449c tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:36:12.711044 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f1ca63a9-1aaa-4f90-99b8-302a6ff5449c tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:36:12.711044 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f1ca63a9-1aaa-4f90-99b8-302a6ff5449c tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:36:12.711271 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f1ca63a9-1aaa-4f90-99b8-302a6ff5449c tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:36:12.711680 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f1ca63a9-1aaa-4f90-99b8-302a6ff5449c tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:36:12.712108 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f1ca63a9-1aaa-4f90-99b8-302a6ff5449c tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:36:12.742361 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-f1ca63a9-1aaa-4f90-99b8-302a6ff5449c tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.048831 Aug 08 02:36:12.742919 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 541/1091] 10.222.0.27 () {58 vars in 1302 bytes} [Mon Aug 8 02:36:12 2022] GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5/os-volume_attachments => generated 25 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Aug 08 02:36:12.759117 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-3e928815-b96b-4eee-b9dd-48d49b22f791 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:36:12.762222 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-3e928815-b96b-4eee-b9dd-48d49b22f791 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] [instance: 0e099ac0-c453-43b0-8bc1-b4c162d483a5] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:36:12.766469 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3e928815-b96b-4eee-b9dd-48d49b22f791 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:36:12.766955 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3e928815-b96b-4eee-b9dd-48d49b22f791 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:36:12.767393 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3e928815-b96b-4eee-b9dd-48d49b22f791 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:36:12.768107 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3e928815-b96b-4eee-b9dd-48d49b22f791 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:36:12.768586 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3e928815-b96b-4eee-b9dd-48d49b22f791 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:36:12.769024 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3e928815-b96b-4eee-b9dd-48d49b22f791 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:36:12.844512 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.wsgi [None req-3e928815-b96b-4eee-b9dd-48d49b22f791 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] HTTP exception thrown: Instance 0e099ac0-c453-43b0-8bc1-b4c162d483a5 is not attached to volume f2ca7843-6d4d-4fc9-99e3-60fb4027b403 Aug 08 02:36:12.847754 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-3e928815-b96b-4eee-b9dd-48d49b22f791 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Request method failure captured: Aug 08 02:36:12.847754 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: request: DELETE /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5/os-volume_attachments/f2ca7843-6d4d-4fc9-99e3-60fb4027b403 HTTP/1.1 Aug 08 02:36:12.847754 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept: application/json Aug 08 02:36:12.847754 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept-Encoding: identity Aug 08 02:36:12.847754 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Connection: close Aug 08 02:36:12.847754 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Length: 0 Aug 08 02:36:12.847754 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Type: application/json Aug 08 02:36:12.847754 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Host: 10.222.0.27 Aug 08 02:36:12.847754 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Openstack-System-Scope: None Aug 08 02:36:12.847754 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: User-Agent: python-urllib3/1.26.11 Aug 08 02:36:12.847754 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Auth-Token: gAAAAABi8HYV4IzvORp7bt7fIAMBUCPWhvRCz4d1mGhZKe41IFVRjJfutbxDEd7W0BGC5FneVlylk8QYo-Mu_9zpL1iOu3ZoFRFlMbTJf3hE6k-feJQrrOzUSnXmntehBdar6ZyvMDhrOlqv1eXtM4ioP40dDOdRHJD_d7wGR0HiXGO4XVA4zB8 Aug 08 02:36:12.847754 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Id: None Aug 08 02:36:12.847754 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Name: None Aug 08 02:36:12.847754 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Identity-Status: Confirmed Aug 08 02:36:12.847754 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Is-Admin-Project: True Aug 08 02:36:12.847754 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Id: default Aug 08 02:36:12.847754 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Name: Default Aug 08 02:36:12.847754 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Id: dc5ea7a1b6a54cd2af5d648f09aaed63 Aug 08 02:36:12.847754 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Name: tempest-TestMinimumBasicScenario-114677538 Aug 08 02:36:12.847754 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Role: reader,member Aug 08 02:36:12.847754 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Roles: reader,member Aug 08 02:36:12.849506 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/dc5ea7a1b6a54cd2af5d648f09aaed63"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/dc5ea7a1b6a54cd2af5d648f09aaed63"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_dc5ea7a1b6a54cd2af5d648f09aaed63", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/dc5ea7a1b6a54cd2af5d648f09aaed63"}]}] Aug 08 02:36:12.849506 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant: tempest-TestMinimumBasicScenario-114677538 Aug 08 02:36:12.849506 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Id: dc5ea7a1b6a54cd2af5d648f09aaed63 Aug 08 02:36:12.849506 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Name: tempest-TestMinimumBasicScenario-114677538 Aug 08 02:36:12.849506 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User: tempest-TestMinimumBasicScenario-114677538-project Aug 08 02:36:12.849506 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Id: default Aug 08 02:36:12.849506 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Name: Default Aug 08 02:36:12.849506 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Id: f5975379433a4e31a242764f58954f43 Aug 08 02:36:12.849506 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Name: tempest-TestMinimumBasicScenario-114677538-project Aug 08 02:36:12.849506 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: method: > Aug 08 02:36:12.849506 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: exception: Instance 0e099ac0-c453-43b0-8bc1-b4c162d483a5 is not attached to volume f2ca7843-6d4d-4fc9-99e3-60fb4027b403 Aug 08 02:36:12.850338 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: action_args: {'server_id': '0e099ac0-c453-43b0-8bc1-b4c162d483a5', 'id': 'f2ca7843-6d4d-4fc9-99e3-60fb4027b403'} Aug 08 02:36:12.850338 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:36:12.850338 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:36:12.850338 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Aug 08 02:36:12.850338 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Aug 08 02:36:12.850338 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Aug 08 02:36:12.850338 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Aug 08 02:36:12.850338 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 281, in get_by_volume_and_instance Aug 08 02:36:12.850338 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Aug 08 02:36:12.850338 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id f2ca7843-6d4d-4fc9-99e3-60fb4027b403. Aug 08 02:36:12.850338 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:36:12.850338 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:36:12.850338 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:36:12.850338 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:36:12.850338 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:36:12.850338 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:36:12.850338 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:36:12.850338 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:36:12.850338 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:36:12.851678 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:36:12.851678 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Aug 08 02:36:12.851678 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Aug 08 02:36:12.851678 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0e099ac0-c453-43b0-8bc1-b4c162d483a5 is not attached to volume f2ca7843-6d4d-4fc9-99e3-60fb4027b403 Aug 08 02:36:12.851678 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:36:12.851678 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:36:12.851678 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:36:12.851678 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:36:12.851678 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:36:12.851678 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:36:12.851678 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:36:12.851678 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:36:12.851678 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0e099ac0-c453-43b0-8bc1-b4c162d483a5 is not attached to volume f2ca7843-6d4d-4fc9-99e3-60fb4027b403 Aug 08 02:36:12.851678 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:36:12.851678 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-3e928815-b96b-4eee-b9dd-48d49b22f791 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Returning 404 to user: Instance 0e099ac0-c453-43b0-8bc1-b4c162d483a5 is not attached to volume f2ca7843-6d4d-4fc9-99e3-60fb4027b403 {{(pid=118352) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:36:12.852899 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-3e928815-b96b-4eee-b9dd-48d49b22f791 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "DELETE /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5/os-volume_attachments/f2ca7843-6d4d-4fc9-99e3-60fb4027b403" status: 404 len: 154 microversion: 2.1 time: 0.094689 Aug 08 02:36:12.852899 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 551/1092] 10.222.0.27 () {58 vars in 1416 bytes} [Mon Aug 8 02:36:12 2022] DELETE /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5/os-volume_attachments/f2ca7843-6d4d-4fc9-99e3-60fb4027b403 => generated 154 bytes in 96 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 08 02:36:14.103380 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-82b58fd3-c825-425c-9895-afeeb3f019ce tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:36:14.103896 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-82b58fd3-c825-425c-9895-afeeb3f019ce tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] [instance: 0e099ac0-c453-43b0-8bc1-b4c162d483a5] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:36:14.110204 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-82b58fd3-c825-425c-9895-afeeb3f019ce tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:36:14.110688 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-82b58fd3-c825-425c-9895-afeeb3f019ce tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:36:14.111118 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-82b58fd3-c825-425c-9895-afeeb3f019ce tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:36:14.111803 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-82b58fd3-c825-425c-9895-afeeb3f019ce tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:36:14.112253 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-82b58fd3-c825-425c-9895-afeeb3f019ce tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:36:14.112689 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-82b58fd3-c825-425c-9895-afeeb3f019ce tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:36:14.137294 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-82b58fd3-c825-425c-9895-afeeb3f019ce tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] [instance: 0e099ac0-c453-43b0-8bc1-b4c162d483a5] Going to try to terminate instance {{(pid=118351) delete /opt/stack/nova/nova/compute/api.py:2666}} Aug 08 02:36:14.367705 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-82b58fd3-c825-425c-9895-afeeb3f019ce tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "DELETE /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5" status: 204 len: 0 microversion: 2.1 time: 0.268537 Aug 08 02:36:14.368219 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 542/1093] 10.222.0.27 () {58 vars in 1239 bytes} [Mon Aug 8 02:36:14 2022] DELETE /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 08 02:36:14.376593 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-246cf26d-a960-4390-b199-820f7811208c tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:36:14.377876 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-246cf26d-a960-4390-b199-820f7811208c tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] [instance: 0e099ac0-c453-43b0-8bc1-b4c162d483a5] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:36:14.383731 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-246cf26d-a960-4390-b199-820f7811208c tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:36:14.384222 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-246cf26d-a960-4390-b199-820f7811208c tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:36:14.384675 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-246cf26d-a960-4390-b199-820f7811208c tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:36:14.385394 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-246cf26d-a960-4390-b199-820f7811208c tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:36:14.385855 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-246cf26d-a960-4390-b199-820f7811208c tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:36:14.386285 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-246cf26d-a960-4390-b199-820f7811208c tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:36:14.516245 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-246cf26d-a960-4390-b199-820f7811208c tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5975379433a4e31a242764f58954f43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc5ea7a1b6a54cd2af5d648f09aaed63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:36:14.524501 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-246cf26d-a960-4390-b199-820f7811208c tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5" status: 200 len: 1555 microversion: 2.1 time: 0.152058 Aug 08 02:36:14.525088 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 552/1094] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:36:14 2022] GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5 => generated 1555 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:36:15.545930 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-0fe9c584-8d43-45dd-aa16-d95d96f694a3 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:36:15.546632 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-0fe9c584-8d43-45dd-aa16-d95d96f694a3 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] [instance: 0e099ac0-c453-43b0-8bc1-b4c162d483a5] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:36:15.553057 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0fe9c584-8d43-45dd-aa16-d95d96f694a3 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:36:15.553548 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0fe9c584-8d43-45dd-aa16-d95d96f694a3 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:36:15.553986 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0fe9c584-8d43-45dd-aa16-d95d96f694a3 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:36:15.554679 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0fe9c584-8d43-45dd-aa16-d95d96f694a3 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:36:15.555139 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0fe9c584-8d43-45dd-aa16-d95d96f694a3 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:36:15.555569 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-0fe9c584-8d43-45dd-aa16-d95d96f694a3 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:36:15.687851 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-0fe9c584-8d43-45dd-aa16-d95d96f694a3 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5975379433a4e31a242764f58954f43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc5ea7a1b6a54cd2af5d648f09aaed63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:36:15.695377 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-0fe9c584-8d43-45dd-aa16-d95d96f694a3 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5" status: 200 len: 1555 microversion: 2.1 time: 0.153701 Aug 08 02:36:15.695944 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 543/1095] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:36:15 2022] GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5 => generated 1555 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:36:16.657423 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [req-79b7e2f9-06a8-4823-b674-ef600f763646 req-31c95771-010d-48d4-a444-e2027cd61b42 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0e099ac0-c453-43b0-8bc1-b4c162d483a5", "name": "network-vif-deleted", "tag": "35cd469b-968e-46d3-8566-8026ce405507"}]} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:36:16.667029 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-79b7e2f9-06a8-4823-b674-ef600f763646 req-31c95771-010d-48d4-a444-e2027cd61b42 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:36:16.667545 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-79b7e2f9-06a8-4823-b674-ef600f763646 req-31c95771-010d-48d4-a444-e2027cd61b42 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:36:16.667978 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-79b7e2f9-06a8-4823-b674-ef600f763646 req-31c95771-010d-48d4-a444-e2027cd61b42 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:36:16.679117 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.compute.server_external_events [req-79b7e2f9-06a8-4823-b674-ef600f763646 req-31c95771-010d-48d4-a444-e2027cd61b42 service nova] Creating event network-vif-deleted:35cd469b-968e-46d3-8566-8026ce405507 for instance 0e099ac0-c453-43b0-8bc1-b4c162d483a5 on hv-cinder-74014 Aug 08 02:36:16.686668 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [req-79b7e2f9-06a8-4823-b674-ef600f763646 req-31c95771-010d-48d4-a444-e2027cd61b42 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.090253 Aug 08 02:36:16.687411 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 553/1096] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:36:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:36:16.717433 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-a60f369c-0750-4c04-9c4a-e4cec433572b tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:36:16.718051 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-a60f369c-0750-4c04-9c4a-e4cec433572b tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] [instance: 0e099ac0-c453-43b0-8bc1-b4c162d483a5] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:36:16.724518 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a60f369c-0750-4c04-9c4a-e4cec433572b tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:36:16.725073 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a60f369c-0750-4c04-9c4a-e4cec433572b tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:36:16.725505 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a60f369c-0750-4c04-9c4a-e4cec433572b tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:36:16.726190 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a60f369c-0750-4c04-9c4a-e4cec433572b tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:36:16.726639 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a60f369c-0750-4c04-9c4a-e4cec433572b tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:36:16.727059 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-a60f369c-0750-4c04-9c4a-e4cec433572b tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:36:16.788770 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-a60f369c-0750-4c04-9c4a-e4cec433572b tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5975379433a4e31a242764f58954f43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc5ea7a1b6a54cd2af5d648f09aaed63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:36:16.801030 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-a60f369c-0750-4c04-9c4a-e4cec433572b tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5" status: 200 len: 1513 microversion: 2.1 time: 0.087347 Aug 08 02:36:16.801612 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 544/1097] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:36:16 2022] GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5 => generated 1513 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:36:17.823055 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-d131c1a5-8d75-4ff0-8269-be2036d786a1 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:36:17.823751 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-d131c1a5-8d75-4ff0-8269-be2036d786a1 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] [instance: 0e099ac0-c453-43b0-8bc1-b4c162d483a5] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:36:17.831641 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d131c1a5-8d75-4ff0-8269-be2036d786a1 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:36:17.832133 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d131c1a5-8d75-4ff0-8269-be2036d786a1 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:36:17.832575 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d131c1a5-8d75-4ff0-8269-be2036d786a1 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:36:17.833457 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d131c1a5-8d75-4ff0-8269-be2036d786a1 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:36:17.833917 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d131c1a5-8d75-4ff0-8269-be2036d786a1 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:36:17.834346 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d131c1a5-8d75-4ff0-8269-be2036d786a1 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:36:17.847460 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.wsgi [None req-d131c1a5-8d75-4ff0-8269-be2036d786a1 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] HTTP exception thrown: Instance 0e099ac0-c453-43b0-8bc1-b4c162d483a5 could not be found. Aug 08 02:36:17.849101 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-d131c1a5-8d75-4ff0-8269-be2036d786a1 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Request method failure captured: Aug 08 02:36:17.849101 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: request: GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5 HTTP/1.1 Aug 08 02:36:17.849101 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept: application/json Aug 08 02:36:17.849101 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept-Encoding: identity Aug 08 02:36:17.849101 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Connection: close Aug 08 02:36:17.849101 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Length: 0 Aug 08 02:36:17.849101 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Type: application/json Aug 08 02:36:17.849101 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Host: 10.222.0.27 Aug 08 02:36:17.849101 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Openstack-System-Scope: None Aug 08 02:36:17.849101 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: User-Agent: python-urllib3/1.26.11 Aug 08 02:36:17.849101 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Auth-Token: gAAAAABi8HYV4IzvORp7bt7fIAMBUCPWhvRCz4d1mGhZKe41IFVRjJfutbxDEd7W0BGC5FneVlylk8QYo-Mu_9zpL1iOu3ZoFRFlMbTJf3hE6k-feJQrrOzUSnXmntehBdar6ZyvMDhrOlqv1eXtM4ioP40dDOdRHJD_d7wGR0HiXGO4XVA4zB8 Aug 08 02:36:17.849101 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Id: None Aug 08 02:36:17.849101 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Name: None Aug 08 02:36:17.849101 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Identity-Status: Confirmed Aug 08 02:36:17.849101 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Is-Admin-Project: True Aug 08 02:36:17.849101 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Id: default Aug 08 02:36:17.849101 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Name: Default Aug 08 02:36:17.849101 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Id: dc5ea7a1b6a54cd2af5d648f09aaed63 Aug 08 02:36:17.849101 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Name: tempest-TestMinimumBasicScenario-114677538 Aug 08 02:36:17.849101 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Role: reader,member Aug 08 02:36:17.849101 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Roles: reader,member Aug 08 02:36:17.850885 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/dc5ea7a1b6a54cd2af5d648f09aaed63"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/dc5ea7a1b6a54cd2af5d648f09aaed63"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_dc5ea7a1b6a54cd2af5d648f09aaed63", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/dc5ea7a1b6a54cd2af5d648f09aaed63"}]}] Aug 08 02:36:17.850885 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant: tempest-TestMinimumBasicScenario-114677538 Aug 08 02:36:17.850885 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Id: dc5ea7a1b6a54cd2af5d648f09aaed63 Aug 08 02:36:17.850885 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Name: tempest-TestMinimumBasicScenario-114677538 Aug 08 02:36:17.850885 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User: tempest-TestMinimumBasicScenario-114677538-project Aug 08 02:36:17.850885 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Id: default Aug 08 02:36:17.850885 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Name: Default Aug 08 02:36:17.850885 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Id: f5975379433a4e31a242764f58954f43 Aug 08 02:36:17.850885 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Name: tempest-TestMinimumBasicScenario-114677538-project Aug 08 02:36:17.850885 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: method: > Aug 08 02:36:17.850885 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: exception: Instance 0e099ac0-c453-43b0-8bc1-b4c162d483a5 could not be found. Aug 08 02:36:17.850885 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: action_args: {'id': '0e099ac0-c453-43b0-8bc1-b4c162d483a5'} Aug 08 02:36:17.851837 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:36:17.851837 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:36:17.851837 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 08 02:36:17.851837 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 08 02:36:17.851837 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 08 02:36:17.851837 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 08 02:36:17.851837 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 08 02:36:17.851837 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 08 02:36:17.851837 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 08 02:36:17.851837 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 08 02:36:17.851837 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0e099ac0-c453-43b0-8bc1-b4c162d483a5 could not be found. Aug 08 02:36:17.851837 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:36:17.851837 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:36:17.851837 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:36:17.851837 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:36:17.851837 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:36:17.851837 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:36:17.851837 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:36:17.851837 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:36:17.851837 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:36:17.853433 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:36:17.853433 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 08 02:36:17.853433 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 08 02:36:17.853433 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 08 02:36:17.853433 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 08 02:36:17.853433 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 08 02:36:17.853433 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:36:17.853433 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0e099ac0-c453-43b0-8bc1-b4c162d483a5 could not be found. Aug 08 02:36:17.853433 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:36:17.853433 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:36:17.853433 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:36:17.853433 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:36:17.853433 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:36:17.853433 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:36:17.853433 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:36:17.853433 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:36:17.853433 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0e099ac0-c453-43b0-8bc1-b4c162d483a5 could not be found. Aug 08 02:36:17.853433 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:36:17.855017 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-d131c1a5-8d75-4ff0-8269-be2036d786a1 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Returning 404 to user: Instance 0e099ac0-c453-43b0-8bc1-b4c162d483a5 could not be found. {{(pid=118352) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:36:17.855017 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-d131c1a5-8d75-4ff0-8269-be2036d786a1 tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5" status: 404 len: 111 microversion: 2.1 time: 0.032258 Aug 08 02:36:17.855017 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 554/1098] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:36:17 2022] GET /compute/v2.1/servers/0e099ac0-c453-43b0-8bc1-b4c162d483a5 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 08 02:36:17.859176 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-53a6225b-1214-46a9-98b8-3892f0184d6b tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] Calling method '.version_select at 0x7efc74657040>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:36:17.968607 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-53a6225b-1214-46a9-98b8-3892f0184d6b tempest-TestMinimumBasicScenario-114677538 tempest-TestMinimumBasicScenario-114677538-project] 10.222.0.27 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-608953587" status: 202 len: 0 microversion: 2.1 time: 0.113148 Aug 08 02:36:17.969220 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 545/1099] 10.222.0.27 () {58 vars in 1269 bytes} [Mon Aug 8 02:36:17 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-608953587 => generated 0 bytes in 114 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 08 02:36:58.777424 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-75780aa5-5755-43d4-9b63-8aff383b7fd1 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:36:58.778023 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-75780aa5-5755-43d4-9b63-8aff383b7fd1 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: f0d7098c-da43-4381-99e9-b4756dde20c8] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:36:58.780887 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_api:SELECT=14 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:58.781544 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell1:SELECT=31,nova_cell1:INSERT=1 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:58.786237 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-75780aa5-5755-43d4-9b63-8aff383b7fd1 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:36:58.786721 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-75780aa5-5755-43d4-9b63-8aff383b7fd1 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:36:58.787156 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-75780aa5-5755-43d4-9b63-8aff383b7fd1 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:36:58.787840 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-75780aa5-5755-43d4-9b63-8aff383b7fd1 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:36:58.788295 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-75780aa5-5755-43d4-9b63-8aff383b7fd1 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:36:58.788748 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-75780aa5-5755-43d4-9b63-8aff383b7fd1 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:36:58.815499 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-75780aa5-5755-43d4-9b63-8aff383b7fd1 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: f0d7098c-da43-4381-99e9-b4756dde20c8] Going to try to terminate instance {{(pid=118352) delete /opt/stack/nova/nova/compute/api.py:2666}} Aug 08 02:36:59.046660 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-75780aa5-5755-43d4-9b63-8aff383b7fd1 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "DELETE /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8" status: 204 len: 0 microversion: 2.1 time: 0.272755 Aug 08 02:36:59.047248 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 555/1100] 10.222.0.27 () {58 vars in 1239 bytes} [Mon Aug 8 02:36:58 2022] DELETE /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 08 02:36:59.057697 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell1:SELECT=40,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:59.058446 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_api:SELECT=20,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:59.063383 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-126d45ac-1452-4bcd-a683-6d847f8b9e00 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:36:59.063999 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-126d45ac-1452-4bcd-a683-6d847f8b9e00 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: f0d7098c-da43-4381-99e9-b4756dde20c8] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:36:59.072388 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-126d45ac-1452-4bcd-a683-6d847f8b9e00 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:36:59.072913 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-126d45ac-1452-4bcd-a683-6d847f8b9e00 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:36:59.073347 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-126d45ac-1452-4bcd-a683-6d847f8b9e00 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:36:59.074038 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-126d45ac-1452-4bcd-a683-6d847f8b9e00 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:36:59.074486 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-126d45ac-1452-4bcd-a683-6d847f8b9e00 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:36:59.074907 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-126d45ac-1452-4bcd-a683-6d847f8b9e00 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:36:59.209144 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-126d45ac-1452-4bcd-a683-6d847f8b9e00 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:36:59.217755 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-126d45ac-1452-4bcd-a683-6d847f8b9e00 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8" status: 200 len: 1579 microversion: 2.1 time: 0.161438 Aug 08 02:36:59.218033 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 546/1101] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:36:59 2022] GET /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8 => generated 1579 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:37:00.239479 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-05111547-182a-4a2e-9921-8b427ce22dc9 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:37:00.240181 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-05111547-182a-4a2e-9921-8b427ce22dc9 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: f0d7098c-da43-4381-99e9-b4756dde20c8] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:37:00.246611 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-05111547-182a-4a2e-9921-8b427ce22dc9 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:37:00.247099 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-05111547-182a-4a2e-9921-8b427ce22dc9 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:37:00.247539 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-05111547-182a-4a2e-9921-8b427ce22dc9 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:37:00.248226 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-05111547-182a-4a2e-9921-8b427ce22dc9 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:37:00.248687 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-05111547-182a-4a2e-9921-8b427ce22dc9 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:37:00.249289 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-05111547-182a-4a2e-9921-8b427ce22dc9 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:37:00.382701 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-05111547-182a-4a2e-9921-8b427ce22dc9 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:37:00.390755 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-05111547-182a-4a2e-9921-8b427ce22dc9 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8" status: 200 len: 1579 microversion: 2.1 time: 0.155517 Aug 08 02:37:00.391309 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 556/1102] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:37:00 2022] GET /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8 => generated 1579 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:37:01.406302 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [req-5f86f2f0-4a35-400f-9a34-ae99f20fb46f req-75c4392d-a5b3-40b6-b902-b33d806b1143 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f0d7098c-da43-4381-99e9-b4756dde20c8", "name": "network-vif-deleted", "tag": "80747251-b558-4d89-ba58-3fbb7af11ff4"}]} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:37:01.412377 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-0e6bf947-6c71-4610-830a-b016890fd102 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:37:01.414832 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-0e6bf947-6c71-4610-830a-b016890fd102 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: f0d7098c-da43-4381-99e9-b4756dde20c8] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:37:01.418706 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-5f86f2f0-4a35-400f-9a34-ae99f20fb46f req-75c4392d-a5b3-40b6-b902-b33d806b1143 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:37:01.420193 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-5f86f2f0-4a35-400f-9a34-ae99f20fb46f req-75c4392d-a5b3-40b6-b902-b33d806b1143 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:37:01.420462 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0e6bf947-6c71-4610-830a-b016890fd102 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:37:01.420799 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-5f86f2f0-4a35-400f-9a34-ae99f20fb46f req-75c4392d-a5b3-40b6-b902-b33d806b1143 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:37:01.421823 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0e6bf947-6c71-4610-830a-b016890fd102 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:37:01.422122 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0e6bf947-6c71-4610-830a-b016890fd102 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:37:01.422804 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0e6bf947-6c71-4610-830a-b016890fd102 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:37:01.423267 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0e6bf947-6c71-4610-830a-b016890fd102 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:37:01.423782 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-0e6bf947-6c71-4610-830a-b016890fd102 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:37:01.433644 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.compute.server_external_events [req-5f86f2f0-4a35-400f-9a34-ae99f20fb46f req-75c4392d-a5b3-40b6-b902-b33d806b1143 service nova] Creating event network-vif-deleted:80747251-b558-4d89-ba58-3fbb7af11ff4 for instance f0d7098c-da43-4381-99e9-b4756dde20c8 on hv-cinder-74014 Aug 08 02:37:01.440381 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [req-5f86f2f0-4a35-400f-9a34-ae99f20fb46f req-75c4392d-a5b3-40b6-b902-b33d806b1143 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038081 Aug 08 02:37:01.441220 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 547/1103] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:37:01 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:37:01.500964 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-0e6bf947-6c71-4610-830a-b016890fd102 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:37:01.508064 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-0e6bf947-6c71-4610-830a-b016890fd102 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8" status: 200 len: 1511 microversion: 2.1 time: 0.099757 Aug 08 02:37:01.508877 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 557/1104] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:37:01 2022] GET /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8 => generated 1511 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:37:02.530599 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-1df256a8-ec62-4979-afed-3aa848607d3c tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:37:02.531437 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-1df256a8-ec62-4979-afed-3aa848607d3c tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: f0d7098c-da43-4381-99e9-b4756dde20c8] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:37:02.538211 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1df256a8-ec62-4979-afed-3aa848607d3c tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:37:02.541079 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1df256a8-ec62-4979-afed-3aa848607d3c tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:37:02.541079 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1df256a8-ec62-4979-afed-3aa848607d3c tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:37:02.541079 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1df256a8-ec62-4979-afed-3aa848607d3c tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:37:02.541079 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1df256a8-ec62-4979-afed-3aa848607d3c tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:37:02.541481 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-1df256a8-ec62-4979-afed-3aa848607d3c tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:37:02.555606 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.wsgi [None req-1df256a8-ec62-4979-afed-3aa848607d3c tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] HTTP exception thrown: Instance f0d7098c-da43-4381-99e9-b4756dde20c8 could not be found. Aug 08 02:37:02.557279 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-1df256a8-ec62-4979-afed-3aa848607d3c tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Request method failure captured: Aug 08 02:37:02.557279 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: request: GET /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8 HTTP/1.1 Aug 08 02:37:02.557279 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Accept: application/json Aug 08 02:37:02.557279 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Accept-Encoding: identity Aug 08 02:37:02.557279 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Connection: close Aug 08 02:37:02.557279 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Content-Length: 0 Aug 08 02:37:02.557279 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Content-Type: application/json Aug 08 02:37:02.557279 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Host: 10.222.0.27 Aug 08 02:37:02.557279 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Openstack-System-Scope: None Aug 08 02:37:02.557279 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: User-Agent: python-urllib3/1.26.11 Aug 08 02:37:02.557279 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Auth-Token: gAAAAABi8HYc3ITkcL195SgamaPJ0kidCrco7MQMBVI60W3dhhxQywziFpjko5BS7_5EtpN1OOpCyROL3y5wbY4pgEtWsgqrbP1z3EwAek9uoasyWggHXtdlr9JFso4IRqI1CEN56WsilnuCNo8We4lcFQ-SzMkX3bk_YaN8OPqytfLXsxqiG4E Aug 08 02:37:02.557279 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Domain-Id: None Aug 08 02:37:02.557279 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Domain-Name: None Aug 08 02:37:02.557279 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Identity-Status: Confirmed Aug 08 02:37:02.557279 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Is-Admin-Project: True Aug 08 02:37:02.557279 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Domain-Id: default Aug 08 02:37:02.557279 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Domain-Name: Default Aug 08 02:37:02.557279 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Id: 4e2c1907fad4499aae599d49e1d2bc80 Aug 08 02:37:02.557279 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Name: tempest-TestVolumeBootPattern-1114932292 Aug 08 02:37:02.557279 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Role: reader,member Aug 08 02:37:02.557279 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Roles: reader,member Aug 08 02:37:02.558978 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/4e2c1907fad4499aae599d49e1d2bc80"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/4e2c1907fad4499aae599d49e1d2bc80"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_4e2c1907fad4499aae599d49e1d2bc80", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/4e2c1907fad4499aae599d49e1d2bc80"}]}] Aug 08 02:37:02.558978 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant: tempest-TestVolumeBootPattern-1114932292 Aug 08 02:37:02.558978 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant-Id: 4e2c1907fad4499aae599d49e1d2bc80 Aug 08 02:37:02.558978 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant-Name: tempest-TestVolumeBootPattern-1114932292 Aug 08 02:37:02.558978 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User: tempest-TestVolumeBootPattern-1114932292-project Aug 08 02:37:02.558978 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Domain-Id: default Aug 08 02:37:02.558978 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Domain-Name: Default Aug 08 02:37:02.558978 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Id: b92e9b1d8c96434086055f58b119ed57 Aug 08 02:37:02.558978 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Name: tempest-TestVolumeBootPattern-1114932292-project Aug 08 02:37:02.558978 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: method: > Aug 08 02:37:02.558978 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: exception: Instance f0d7098c-da43-4381-99e9-b4756dde20c8 could not be found. Aug 08 02:37:02.558978 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: action_args: {'id': 'f0d7098c-da43-4381-99e9-b4756dde20c8'} Aug 08 02:37:02.562131 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:37:02.562131 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:37:02.562131 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 08 02:37:02.562131 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 08 02:37:02.562131 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 08 02:37:02.562131 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 08 02:37:02.562131 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 08 02:37:02.562131 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 08 02:37:02.562131 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 08 02:37:02.562131 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 08 02:37:02.562131 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f0d7098c-da43-4381-99e9-b4756dde20c8 could not be found. Aug 08 02:37:02.562131 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:37:02.562131 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:37:02.562131 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:37:02.562131 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:37:02.562131 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:37:02.562131 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:37:02.562131 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:37:02.562131 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:37:02.562131 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:37:02.563553 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:37:02.563553 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 08 02:37:02.563553 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 08 02:37:02.563553 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 08 02:37:02.563553 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 08 02:37:02.563553 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 08 02:37:02.563553 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:37:02.563553 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f0d7098c-da43-4381-99e9-b4756dde20c8 could not be found. Aug 08 02:37:02.563553 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:37:02.563553 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:37:02.563553 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:37:02.563553 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:37:02.563553 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:37:02.563553 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:37:02.563553 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:37:02.563553 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:37:02.563553 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f0d7098c-da43-4381-99e9-b4756dde20c8 could not be found. Aug 08 02:37:02.563553 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:37:02.570100 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-1df256a8-ec62-4979-afed-3aa848607d3c tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Returning 404 to user: Instance f0d7098c-da43-4381-99e9-b4756dde20c8 could not be found. {{(pid=118351) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:37:02.570100 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-1df256a8-ec62-4979-afed-3aa848607d3c tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8" status: 404 len: 111 microversion: 2.1 time: 0.032500 Aug 08 02:37:02.570100 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 548/1105] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:37:02 2022] GET /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 08 02:37:02.659197 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-7071326a-db46-413b-8fec-b318a834abff tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-330907875", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1683228822", "security_groups": [{"name": "tempest-secgroup-smoke-1655674036"}], "block_device_mapping_v2": [{"uuid": "8d756c2b-af46-4d5c-a12f-63de84861b78", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "b0ff42d3-c706-479c-b637-24fa43da993a"}]}} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:37:02.771801 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.network.neutron [None req-7071326a-db46-413b-8fec-b318a834abff tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] validate_networks() for [('b0ff42d3-c706-479c-b637-24fa43da993a', None, None, None, None, None)] {{(pid=118352) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 08 02:37:03.239320 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-7071326a-db46-413b-8fec-b318a834abff tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Flavor limits 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 08 02:37:03.239749 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-7071326a-db46-413b-8fec-b318a834abff tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Image limits 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 08 02:37:03.240227 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-7071326a-db46-413b-8fec-b318a834abff tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Flavor pref 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 08 02:37:03.240684 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-7071326a-db46-413b-8fec-b318a834abff tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Image pref 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 08 02:37:03.241190 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-7071326a-db46-413b-8fec-b318a834abff tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 08 02:37:03.249989 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7071326a-db46-413b-8fec-b318a834abff tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:37:03.250467 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7071326a-db46-413b-8fec-b318a834abff tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:37:03.250905 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7071326a-db46-413b-8fec-b318a834abff tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:37:03.251686 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7071326a-db46-413b-8fec-b318a834abff tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:37:03.252151 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7071326a-db46-413b-8fec-b318a834abff tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:37:03.252581 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-7071326a-db46-413b-8fec-b318a834abff tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:37:03.270069 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.quota [None req-7071326a-db46-413b-8fec-b318a834abff tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Getting quotas for project 4e2c1907fad4499aae599d49e1d2bc80. Resources: {'ram', 'cores', 'instances'} {{(pid=118352) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 08 02:37:03.274378 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.quota [None req-7071326a-db46-413b-8fec-b318a834abff tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Getting quotas for user b92e9b1d8c96434086055f58b119ed57 and project 4e2c1907fad4499aae599d49e1d2bc80. Resources: {'ram', 'cores', 'instances'} {{(pid=118352) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 08 02:37:03.331667 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-7071326a-db46-413b-8fec-b318a834abff tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Going to run 1 instances... {{(pid=118352) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Aug 08 02:37:03.333805 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-7071326a-db46-413b-8fec-b318a834abff tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: fa7f9e9e-8095-49ae-aaab-363a38be7e81] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='8d756c2b-af46-4d5c-a12f-63de84861b78',volume_size=None,volume_type=None)] {{(pid=118352) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Aug 08 02:37:03.745343 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-7071326a-db46-413b-8fec-b318a834abff tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.089988 Aug 08 02:37:03.746302 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 558/1106] 10.222.0.27 () {60 vars in 1147 bytes} [Mon Aug 8 02:37:02 2022] POST /compute/v2.1/servers => generated 406 bytes in 1091 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 08 02:37:03.757543 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-d305c432-ad89-4180-b4e1-b710df2c4b4e tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:37:03.758161 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-d305c432-ad89-4180-b4e1-b710df2c4b4e tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: fa7f9e9e-8095-49ae-aaab-363a38be7e81] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:37:03.816322 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-d305c432-ad89-4180-b4e1-b710df2c4b4e tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:37:03.823087 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-d305c432-ad89-4180-b4e1-b710df2c4b4e tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81" status: 200 len: 1094 microversion: 2.1 time: 0.069405 Aug 08 02:37:03.823640 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 549/1107] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:37:03 2022] GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81 => generated 1094 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:37:04.845369 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-84cf27d0-fb86-4b4a-95f2-d7851c8959bc tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:37:04.846049 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-84cf27d0-fb86-4b4a-95f2-d7851c8959bc tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: fa7f9e9e-8095-49ae-aaab-363a38be7e81] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:37:04.898356 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-84cf27d0-fb86-4b4a-95f2-d7851c8959bc tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:37:04.906511 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-84cf27d0-fb86-4b4a-95f2-d7851c8959bc tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81" status: 200 len: 1094 microversion: 2.1 time: 0.064913 Aug 08 02:37:04.907097 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 559/1108] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:37:04 2022] GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81 => generated 1094 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:37:05.801825 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 08 02:37:05.802046 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Mon Aug 8 02:37:05 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 08 02:37:05.927982 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-81f50ac6-94ef-4635-ab1e-86bff7a4d52b tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:37:05.928518 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-81f50ac6-94ef-4635-ab1e-86bff7a4d52b tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: fa7f9e9e-8095-49ae-aaab-363a38be7e81] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:37:05.982957 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-81f50ac6-94ef-4635-ab1e-86bff7a4d52b tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:37:05.990282 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-81f50ac6-94ef-4635-ab1e-86bff7a4d52b tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81" status: 200 len: 1094 microversion: 2.1 time: 0.066433 Aug 08 02:37:05.990818 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 550/1109] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:37:05 2022] GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81 => generated 1094 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:37:07.011326 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-e3fa7f63-525e-4591-a9ca-2b9d31f20c5b tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:37:07.011932 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-e3fa7f63-525e-4591-a9ca-2b9d31f20c5b tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: fa7f9e9e-8095-49ae-aaab-363a38be7e81] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:37:07.065311 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-e3fa7f63-525e-4591-a9ca-2b9d31f20c5b tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:37:07.071911 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-e3fa7f63-525e-4591-a9ca-2b9d31f20c5b tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81" status: 200 len: 1094 microversion: 2.1 time: 0.064548 Aug 08 02:37:07.072464 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 560/1110] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:37:07 2022] GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81 => generated 1094 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:37:08.092807 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-f4e07b06-2cbe-4469-ab4d-7da908b65739 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:37:08.093478 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-f4e07b06-2cbe-4469-ab4d-7da908b65739 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: fa7f9e9e-8095-49ae-aaab-363a38be7e81] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:37:08.100006 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f4e07b06-2cbe-4469-ab4d-7da908b65739 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:37:08.100476 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f4e07b06-2cbe-4469-ab4d-7da908b65739 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:37:08.100949 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f4e07b06-2cbe-4469-ab4d-7da908b65739 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:37:08.101643 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f4e07b06-2cbe-4469-ab4d-7da908b65739 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:37:08.102103 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f4e07b06-2cbe-4469-ab4d-7da908b65739 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:37:08.102530 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f4e07b06-2cbe-4469-ab4d-7da908b65739 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:37:08.165227 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-f4e07b06-2cbe-4469-ab4d-7da908b65739 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:37:08.172793 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-f4e07b06-2cbe-4469-ab4d-7da908b65739 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81" status: 200 len: 1144 microversion: 2.1 time: 0.083945 Aug 08 02:37:08.173356 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 551/1111] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:37:08 2022] GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81 => generated 1144 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:37:09.192962 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-b12e2640-867d-4dc0-a05e-eebe4042b65e tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:37:09.193576 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-b12e2640-867d-4dc0-a05e-eebe4042b65e tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: fa7f9e9e-8095-49ae-aaab-363a38be7e81] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:37:09.200101 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b12e2640-867d-4dc0-a05e-eebe4042b65e tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:37:09.200589 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b12e2640-867d-4dc0-a05e-eebe4042b65e tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:37:09.201320 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b12e2640-867d-4dc0-a05e-eebe4042b65e tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:37:09.202018 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b12e2640-867d-4dc0-a05e-eebe4042b65e tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:37:09.202472 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b12e2640-867d-4dc0-a05e-eebe4042b65e tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:37:09.203543 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b12e2640-867d-4dc0-a05e-eebe4042b65e tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:37:09.264872 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-b12e2640-867d-4dc0-a05e-eebe4042b65e tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:37:09.272850 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-b12e2640-867d-4dc0-a05e-eebe4042b65e tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81" status: 200 len: 1192 microversion: 2.1 time: 0.083830 Aug 08 02:37:09.273403 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 561/1112] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:37:09 2022] GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81 => generated 1192 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:37:10.294443 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-3c473039-ddae-42e6-bc1e-8df100fa523e tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:37:10.295099 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-3c473039-ddae-42e6-bc1e-8df100fa523e tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: fa7f9e9e-8095-49ae-aaab-363a38be7e81] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:37:10.302467 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3c473039-ddae-42e6-bc1e-8df100fa523e tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:37:10.303175 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3c473039-ddae-42e6-bc1e-8df100fa523e tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:37:10.303617 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3c473039-ddae-42e6-bc1e-8df100fa523e tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:37:10.304320 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3c473039-ddae-42e6-bc1e-8df100fa523e tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:37:10.304816 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3c473039-ddae-42e6-bc1e-8df100fa523e tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:37:10.305245 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3c473039-ddae-42e6-bc1e-8df100fa523e tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:37:10.368221 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-3c473039-ddae-42e6-bc1e-8df100fa523e tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:37:10.376149 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-3c473039-ddae-42e6-bc1e-8df100fa523e tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81" status: 200 len: 1210 microversion: 2.1 time: 0.085694 Aug 08 02:37:10.376743 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 552/1113] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:37:10 2022] GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81 => generated 1210 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:37:11.396970 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-f9d5911a-8f6c-41c9-8797-42824c441881 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:37:11.397610 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-f9d5911a-8f6c-41c9-8797-42824c441881 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: fa7f9e9e-8095-49ae-aaab-363a38be7e81] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:37:11.404420 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f9d5911a-8f6c-41c9-8797-42824c441881 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:37:11.414922 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f9d5911a-8f6c-41c9-8797-42824c441881 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:37:11.414922 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f9d5911a-8f6c-41c9-8797-42824c441881 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:37:11.415146 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [req-85535cf2-034a-441a-9304-d80878492e08 req-dacb7234-ead6-43eb-a161-b6468c27f601 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fa7f9e9e-8095-49ae-aaab-363a38be7e81", "tag": "4d9e69b4-4e91-42c2-8eab-03606390a0b6"}]} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:37:11.415599 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f9d5911a-8f6c-41c9-8797-42824c441881 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:37:11.415599 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f9d5911a-8f6c-41c9-8797-42824c441881 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:37:11.415599 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-f9d5911a-8f6c-41c9-8797-42824c441881 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:37:11.416123 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-85535cf2-034a-441a-9304-d80878492e08 req-dacb7234-ead6-43eb-a161-b6468c27f601 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:37:11.416847 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-85535cf2-034a-441a-9304-d80878492e08 req-dacb7234-ead6-43eb-a161-b6468c27f601 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:37:11.417286 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-85535cf2-034a-441a-9304-d80878492e08 req-dacb7234-ead6-43eb-a161-b6468c27f601 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:37:11.429546 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.compute.server_external_events [req-85535cf2-034a-441a-9304-d80878492e08 req-dacb7234-ead6-43eb-a161-b6468c27f601 service nova] Creating event network-changed:4d9e69b4-4e91-42c2-8eab-03606390a0b6 for instance fa7f9e9e-8095-49ae-aaab-363a38be7e81 on hv-cinder-74014 Aug 08 02:37:11.438512 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [req-85535cf2-034a-441a-9304-d80878492e08 req-dacb7234-ead6-43eb-a161-b6468c27f601 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035516 Aug 08 02:37:11.439404 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 553/1114] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:37:11 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 08 02:37:11.543852 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-f9d5911a-8f6c-41c9-8797-42824c441881 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:37:11.552084 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-f9d5911a-8f6c-41c9-8797-42824c441881 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81" status: 200 len: 1266 microversion: 2.1 time: 0.158893 Aug 08 02:37:11.552644 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 562/1115] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:37:11 2022] GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81 => generated 1266 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:37:12.574391 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-e1e1be02-4559-4e7d-8232-2d0998fc40d8 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:37:12.575198 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-e1e1be02-4559-4e7d-8232-2d0998fc40d8 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: fa7f9e9e-8095-49ae-aaab-363a38be7e81] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:37:12.581682 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e1e1be02-4559-4e7d-8232-2d0998fc40d8 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:37:12.582165 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e1e1be02-4559-4e7d-8232-2d0998fc40d8 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:37:12.582601 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e1e1be02-4559-4e7d-8232-2d0998fc40d8 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:37:12.583289 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e1e1be02-4559-4e7d-8232-2d0998fc40d8 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:37:12.583748 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e1e1be02-4559-4e7d-8232-2d0998fc40d8 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:37:12.584172 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-e1e1be02-4559-4e7d-8232-2d0998fc40d8 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:37:12.716202 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-e1e1be02-4559-4e7d-8232-2d0998fc40d8 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:37:12.724419 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-e1e1be02-4559-4e7d-8232-2d0998fc40d8 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81" status: 200 len: 1266 microversion: 2.1 time: 0.154723 Aug 08 02:37:12.725006 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 554/1116] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:37:12 2022] GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81 => generated 1266 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:37:12.762350 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 08 02:37:12.762534 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Mon Aug 8 02:37:12 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 08 02:37:13.745844 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-9abbba76-6639-47ff-8859-430efe06eb0e tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:37:13.746496 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-9abbba76-6639-47ff-8859-430efe06eb0e tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: fa7f9e9e-8095-49ae-aaab-363a38be7e81] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:37:13.748537 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell0:SELECT=2 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:37:13.754346 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9abbba76-6639-47ff-8859-430efe06eb0e tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:37:13.754823 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9abbba76-6639-47ff-8859-430efe06eb0e tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:37:13.755257 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9abbba76-6639-47ff-8859-430efe06eb0e tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:37:13.755967 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9abbba76-6639-47ff-8859-430efe06eb0e tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:37:13.756413 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9abbba76-6639-47ff-8859-430efe06eb0e tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:37:13.756869 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-9abbba76-6639-47ff-8859-430efe06eb0e tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:37:13.886393 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-9abbba76-6639-47ff-8859-430efe06eb0e tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:37:13.894122 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-9abbba76-6639-47ff-8859-430efe06eb0e tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81" status: 200 len: 1266 microversion: 2.1 time: 0.152132 Aug 08 02:37:13.894684 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 563/1117] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:37:13 2022] GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81 => generated 1266 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:37:14.914927 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-2e92df6a-9896-4485-8410-9fb07511c8f1 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:37:14.915573 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-2e92df6a-9896-4485-8410-9fb07511c8f1 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: fa7f9e9e-8095-49ae-aaab-363a38be7e81] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:37:14.922130 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2e92df6a-9896-4485-8410-9fb07511c8f1 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:37:14.922617 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2e92df6a-9896-4485-8410-9fb07511c8f1 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:37:14.923055 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2e92df6a-9896-4485-8410-9fb07511c8f1 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:37:14.923743 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2e92df6a-9896-4485-8410-9fb07511c8f1 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:37:14.924198 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2e92df6a-9896-4485-8410-9fb07511c8f1 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:37:14.924625 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-2e92df6a-9896-4485-8410-9fb07511c8f1 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:37:15.268896 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-2e92df6a-9896-4485-8410-9fb07511c8f1 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:37:15.276417 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-2e92df6a-9896-4485-8410-9fb07511c8f1 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81" status: 200 len: 1266 microversion: 2.1 time: 0.365368 Aug 08 02:37:15.277007 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 555/1118] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:37:14 2022] GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81 => generated 1266 bytes in 367 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:37:16.297705 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-512cfc04-9d77-4782-b26d-cbe73cec210f tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:37:16.298882 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-512cfc04-9d77-4782-b26d-cbe73cec210f tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: fa7f9e9e-8095-49ae-aaab-363a38be7e81] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:37:16.304848 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-512cfc04-9d77-4782-b26d-cbe73cec210f tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:37:16.305334 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-512cfc04-9d77-4782-b26d-cbe73cec210f tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:37:16.305772 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-512cfc04-9d77-4782-b26d-cbe73cec210f tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:37:16.306464 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-512cfc04-9d77-4782-b26d-cbe73cec210f tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:37:16.306921 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-512cfc04-9d77-4782-b26d-cbe73cec210f tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:37:16.307352 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-512cfc04-9d77-4782-b26d-cbe73cec210f tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:37:16.438676 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-512cfc04-9d77-4782-b26d-cbe73cec210f tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:37:16.446496 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-512cfc04-9d77-4782-b26d-cbe73cec210f tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81" status: 200 len: 1266 microversion: 2.1 time: 0.152673 Aug 08 02:37:16.447085 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 564/1119] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:37:16 2022] GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81 => generated 1266 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:37:17.467481 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-867a1766-0ff6-4448-82c4-25d5bee518d2 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:37:17.468116 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-867a1766-0ff6-4448-82c4-25d5bee518d2 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: fa7f9e9e-8095-49ae-aaab-363a38be7e81] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:37:17.471582 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell0:SELECT=4 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:37:17.476591 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-867a1766-0ff6-4448-82c4-25d5bee518d2 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:37:17.477107 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-867a1766-0ff6-4448-82c4-25d5bee518d2 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:37:17.477593 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-867a1766-0ff6-4448-82c4-25d5bee518d2 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:37:17.478306 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-867a1766-0ff6-4448-82c4-25d5bee518d2 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:37:17.478793 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-867a1766-0ff6-4448-82c4-25d5bee518d2 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:37:17.479213 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-867a1766-0ff6-4448-82c4-25d5bee518d2 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:37:17.615791 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-867a1766-0ff6-4448-82c4-25d5bee518d2 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:37:17.622656 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-867a1766-0ff6-4448-82c4-25d5bee518d2 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81" status: 200 len: 1266 microversion: 2.1 time: 0.159224 Aug 08 02:37:17.623203 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 556/1120] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:37:17 2022] GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81 => generated 1266 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:37:18.643471 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-79781ecf-edd7-41d3-b042-14743cfface3 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:37:18.644081 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-79781ecf-edd7-41d3-b042-14743cfface3 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: fa7f9e9e-8095-49ae-aaab-363a38be7e81] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:37:18.645979 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell0:SELECT=10 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:37:18.651235 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-79781ecf-edd7-41d3-b042-14743cfface3 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:37:18.654637 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-79781ecf-edd7-41d3-b042-14743cfface3 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:37:18.654637 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-79781ecf-edd7-41d3-b042-14743cfface3 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:37:18.654637 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-79781ecf-edd7-41d3-b042-14743cfface3 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:37:18.654637 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-79781ecf-edd7-41d3-b042-14743cfface3 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:37:18.655366 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-79781ecf-edd7-41d3-b042-14743cfface3 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:37:18.785650 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-79781ecf-edd7-41d3-b042-14743cfface3 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:37:18.793063 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-79781ecf-edd7-41d3-b042-14743cfface3 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81" status: 200 len: 1266 microversion: 2.1 time: 0.153505 Aug 08 02:37:18.793641 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 565/1121] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:37:18 2022] GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81 => generated 1266 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:37:19.814156 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-ee921839-80af-44f3-bcc5-6652db84f0fa tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:37:19.815497 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-ee921839-80af-44f3-bcc5-6652db84f0fa tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: fa7f9e9e-8095-49ae-aaab-363a38be7e81] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:37:19.821614 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ee921839-80af-44f3-bcc5-6652db84f0fa tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:37:19.822098 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ee921839-80af-44f3-bcc5-6652db84f0fa tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:37:19.822539 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ee921839-80af-44f3-bcc5-6652db84f0fa tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:37:19.823224 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ee921839-80af-44f3-bcc5-6652db84f0fa tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:37:19.823677 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ee921839-80af-44f3-bcc5-6652db84f0fa tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:37:19.824101 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-ee921839-80af-44f3-bcc5-6652db84f0fa tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:37:19.953667 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-ee921839-80af-44f3-bcc5-6652db84f0fa tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:37:19.961310 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-ee921839-80af-44f3-bcc5-6652db84f0fa tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81" status: 200 len: 1266 microversion: 2.1 time: 0.151157 Aug 08 02:37:19.961870 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 557/1122] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:37:19 2022] GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81 => generated 1266 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:37:20.982461 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-6f2c185b-ef80-41a3-91ad-d78cfc99c7dd tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:37:20.982991 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-6f2c185b-ef80-41a3-91ad-d78cfc99c7dd tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: fa7f9e9e-8095-49ae-aaab-363a38be7e81] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:37:20.990055 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6f2c185b-ef80-41a3-91ad-d78cfc99c7dd tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:37:20.990536 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6f2c185b-ef80-41a3-91ad-d78cfc99c7dd tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:37:20.990969 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6f2c185b-ef80-41a3-91ad-d78cfc99c7dd tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:37:20.991655 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6f2c185b-ef80-41a3-91ad-d78cfc99c7dd tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:37:20.992108 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6f2c185b-ef80-41a3-91ad-d78cfc99c7dd tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:37:20.992531 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-6f2c185b-ef80-41a3-91ad-d78cfc99c7dd tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:37:21.128625 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-6f2c185b-ef80-41a3-91ad-d78cfc99c7dd tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:37:21.136123 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-6f2c185b-ef80-41a3-91ad-d78cfc99c7dd tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81" status: 200 len: 1266 microversion: 2.1 time: 0.157606 Aug 08 02:37:21.136908 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 566/1123] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:37:20 2022] GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81 => generated 1266 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:37:22.156645 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-648b5d04-d8db-43e7-b6f8-947031528bce tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:37:22.157285 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-648b5d04-d8db-43e7-b6f8-947031528bce tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: fa7f9e9e-8095-49ae-aaab-363a38be7e81] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:37:22.164322 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-648b5d04-d8db-43e7-b6f8-947031528bce tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:37:22.164853 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-648b5d04-d8db-43e7-b6f8-947031528bce tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:37:22.165307 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-648b5d04-d8db-43e7-b6f8-947031528bce tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:37:22.165998 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-648b5d04-d8db-43e7-b6f8-947031528bce tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:37:22.166459 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-648b5d04-d8db-43e7-b6f8-947031528bce tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:37:22.166889 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-648b5d04-d8db-43e7-b6f8-947031528bce tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:37:22.310345 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-648b5d04-d8db-43e7-b6f8-947031528bce tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:37:22.317985 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-648b5d04-d8db-43e7-b6f8-947031528bce tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81" status: 200 len: 1266 microversion: 2.1 time: 0.165224 Aug 08 02:37:22.318564 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 558/1124] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:37:22 2022] GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81 => generated 1266 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:37:23.340061 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-fc9d016d-5527-47ee-86a6-8fa1f31856bc tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:37:23.340885 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-fc9d016d-5527-47ee-86a6-8fa1f31856bc tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: fa7f9e9e-8095-49ae-aaab-363a38be7e81] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:37:23.348611 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-fc9d016d-5527-47ee-86a6-8fa1f31856bc tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:37:23.349142 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-fc9d016d-5527-47ee-86a6-8fa1f31856bc tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:37:23.349586 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-fc9d016d-5527-47ee-86a6-8fa1f31856bc tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:37:23.350295 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-fc9d016d-5527-47ee-86a6-8fa1f31856bc tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:37:23.350872 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-fc9d016d-5527-47ee-86a6-8fa1f31856bc tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:37:23.351311 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-fc9d016d-5527-47ee-86a6-8fa1f31856bc tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:37:23.496946 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-fc9d016d-5527-47ee-86a6-8fa1f31856bc tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:37:23.504717 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-fc9d016d-5527-47ee-86a6-8fa1f31856bc tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81" status: 200 len: 1266 microversion: 2.1 time: 0.168591 Aug 08 02:37:23.505302 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 567/1125] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:37:23 2022] GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81 => generated 1266 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:37:23.592860 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [req-5cad25be-9e10-4a8c-b11e-ac15186fe233 req-18a3f1ee-eb49-455b-8dfc-269a5c2d690c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fa7f9e9e-8095-49ae-aaab-363a38be7e81", "name": "network-vif-plugged", "status": "completed", "tag": "4d9e69b4-4e91-42c2-8eab-03606390a0b6"}]} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:37:23.599223 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-5cad25be-9e10-4a8c-b11e-ac15186fe233 req-18a3f1ee-eb49-455b-8dfc-269a5c2d690c service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:37:23.600018 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-5cad25be-9e10-4a8c-b11e-ac15186fe233 req-18a3f1ee-eb49-455b-8dfc-269a5c2d690c service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:37:23.601268 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-5cad25be-9e10-4a8c-b11e-ac15186fe233 req-18a3f1ee-eb49-455b-8dfc-269a5c2d690c service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:37:23.612799 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.compute.server_external_events [req-5cad25be-9e10-4a8c-b11e-ac15186fe233 req-18a3f1ee-eb49-455b-8dfc-269a5c2d690c service nova] Creating event network-vif-plugged:4d9e69b4-4e91-42c2-8eab-03606390a0b6 for instance fa7f9e9e-8095-49ae-aaab-363a38be7e81 on hv-cinder-74014 Aug 08 02:37:23.618145 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [req-5cad25be-9e10-4a8c-b11e-ac15186fe233 req-18a3f1ee-eb49-455b-8dfc-269a5c2d690c service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.158939 Aug 08 02:37:23.618782 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 559/1126] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:37:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:37:24.526693 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-e347804b-1749-42b5-8948-e771263ec785 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:37:24.527311 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-e347804b-1749-42b5-8948-e771263ec785 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: fa7f9e9e-8095-49ae-aaab-363a38be7e81] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:37:24.534366 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e347804b-1749-42b5-8948-e771263ec785 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:37:24.534845 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e347804b-1749-42b5-8948-e771263ec785 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:37:24.535285 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e347804b-1749-42b5-8948-e771263ec785 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:37:24.535975 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e347804b-1749-42b5-8948-e771263ec785 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:37:24.536429 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e347804b-1749-42b5-8948-e771263ec785 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:37:24.536883 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-e347804b-1749-42b5-8948-e771263ec785 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:37:24.665418 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-e347804b-1749-42b5-8948-e771263ec785 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:37:24.672854 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-e347804b-1749-42b5-8948-e771263ec785 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81" status: 200 len: 1266 microversion: 2.1 time: 0.150043 Aug 08 02:37:24.673422 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 568/1127] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:37:24 2022] GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81 => generated 1266 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:37:25.695565 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-b16a0d6e-c345-4a3c-b1bf-9e3a0ed3f7ea tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:37:25.696316 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-b16a0d6e-c345-4a3c-b1bf-9e3a0ed3f7ea tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: fa7f9e9e-8095-49ae-aaab-363a38be7e81] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:37:25.702806 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b16a0d6e-c345-4a3c-b1bf-9e3a0ed3f7ea tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:37:25.703290 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b16a0d6e-c345-4a3c-b1bf-9e3a0ed3f7ea tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:37:25.703727 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b16a0d6e-c345-4a3c-b1bf-9e3a0ed3f7ea tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:37:25.704420 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b16a0d6e-c345-4a3c-b1bf-9e3a0ed3f7ea tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:37:25.705446 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b16a0d6e-c345-4a3c-b1bf-9e3a0ed3f7ea tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:37:25.706013 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-b16a0d6e-c345-4a3c-b1bf-9e3a0ed3f7ea tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:37:25.838968 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-b16a0d6e-c345-4a3c-b1bf-9e3a0ed3f7ea tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:37:25.847185 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-b16a0d6e-c345-4a3c-b1bf-9e3a0ed3f7ea tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81" status: 200 len: 1453 microversion: 2.1 time: 0.156333 Aug 08 02:37:25.847746 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 560/1128] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:37:25 2022] GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81 => generated 1453 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 08 02:37:25.867577 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-ef2b21ff-ac04-4df7-ad3e-d33631617903 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:37:25.868191 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-ef2b21ff-ac04-4df7-ad3e-d33631617903 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: fa7f9e9e-8095-49ae-aaab-363a38be7e81] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:37:25.874840 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ef2b21ff-ac04-4df7-ad3e-d33631617903 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:37:25.875327 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ef2b21ff-ac04-4df7-ad3e-d33631617903 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:37:25.875764 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ef2b21ff-ac04-4df7-ad3e-d33631617903 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:37:25.876462 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ef2b21ff-ac04-4df7-ad3e-d33631617903 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:37:25.877659 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ef2b21ff-ac04-4df7-ad3e-d33631617903 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:37:25.878094 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-ef2b21ff-ac04-4df7-ad3e-d33631617903 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:37:26.009922 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-ef2b21ff-ac04-4df7-ad3e-d33631617903 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:37:26.017516 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-ef2b21ff-ac04-4df7-ad3e-d33631617903 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81" status: 200 len: 1453 microversion: 2.1 time: 0.153721 Aug 08 02:37:26.018281 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 569/1129] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:37:25 2022] GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81 => generated 1453 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:37:27.950146 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [req-c8b6bb8f-aceb-435d-afa4-6bcfc83f7917 req-9b8c0689-3f30-44d6-aace-63003726f842 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fa7f9e9e-8095-49ae-aaab-363a38be7e81", "tag": "4d9e69b4-4e91-42c2-8eab-03606390a0b6"}]} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:37:27.960301 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-c8b6bb8f-aceb-435d-afa4-6bcfc83f7917 req-9b8c0689-3f30-44d6-aace-63003726f842 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:37:27.960801 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-c8b6bb8f-aceb-435d-afa4-6bcfc83f7917 req-9b8c0689-3f30-44d6-aace-63003726f842 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:37:27.961232 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-c8b6bb8f-aceb-435d-afa4-6bcfc83f7917 req-9b8c0689-3f30-44d6-aace-63003726f842 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:37:27.972741 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.compute.server_external_events [req-c8b6bb8f-aceb-435d-afa4-6bcfc83f7917 req-9b8c0689-3f30-44d6-aace-63003726f842 service nova] Creating event network-changed:4d9e69b4-4e91-42c2-8eab-03606390a0b6 for instance fa7f9e9e-8095-49ae-aaab-363a38be7e81 on hv-cinder-74014 Aug 08 02:37:27.977750 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [req-c8b6bb8f-aceb-435d-afa4-6bcfc83f7917 req-9b8c0689-3f30-44d6-aace-63003726f842 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.086756 Aug 08 02:37:27.978327 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 561/1130] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:37:27 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:39:03.455555 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_api:SELECT=52,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:39:03.456758 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell1:SELECT=82,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:39:03.460195 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-2eac83c9-cb64-4223-b735-b5cdb4cfaed7 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-2104204537", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1683228822", "security_groups": [{"name": "tempest-secgroup-smoke-1655674036"}], "block_device_mapping_v2": [{"uuid": "133cfdea-39a1-4b23-8385-48901abd629e", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "b0ff42d3-c706-479c-b637-24fa43da993a"}]}} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:39:03.571609 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.network.neutron [None req-2eac83c9-cb64-4223-b735-b5cdb4cfaed7 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] validate_networks() for [('b0ff42d3-c706-479c-b637-24fa43da993a', None, None, None, None, None)] {{(pid=118352) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 08 02:39:03.939611 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-2eac83c9-cb64-4223-b735-b5cdb4cfaed7 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Flavor limits 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 08 02:39:03.940040 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-2eac83c9-cb64-4223-b735-b5cdb4cfaed7 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Image limits 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 08 02:39:03.940567 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-2eac83c9-cb64-4223-b735-b5cdb4cfaed7 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Flavor pref 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 08 02:39:03.941021 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-2eac83c9-cb64-4223-b735-b5cdb4cfaed7 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Image pref 0:0:0 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 08 02:39:03.941434 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.virt.hardware [None req-2eac83c9-cb64-4223-b735-b5cdb4cfaed7 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118352) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 08 02:39:03.950123 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2eac83c9-cb64-4223-b735-b5cdb4cfaed7 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:39:03.951131 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2eac83c9-cb64-4223-b735-b5cdb4cfaed7 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:39:03.951576 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2eac83c9-cb64-4223-b735-b5cdb4cfaed7 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:39:03.952298 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2eac83c9-cb64-4223-b735-b5cdb4cfaed7 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:39:03.953043 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2eac83c9-cb64-4223-b735-b5cdb4cfaed7 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:39:03.953472 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-2eac83c9-cb64-4223-b735-b5cdb4cfaed7 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:39:03.971878 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.quota [None req-2eac83c9-cb64-4223-b735-b5cdb4cfaed7 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Getting quotas for project 4e2c1907fad4499aae599d49e1d2bc80. Resources: {'ram', 'cores', 'instances'} {{(pid=118352) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 08 02:39:03.975642 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.quota [None req-2eac83c9-cb64-4223-b735-b5cdb4cfaed7 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Getting quotas for user b92e9b1d8c96434086055f58b119ed57 and project 4e2c1907fad4499aae599d49e1d2bc80. Resources: {'ram', 'cores', 'instances'} {{(pid=118352) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 08 02:39:04.032386 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-2eac83c9-cb64-4223-b735-b5cdb4cfaed7 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Going to run 1 instances... {{(pid=118352) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Aug 08 02:39:04.034520 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-2eac83c9-cb64-4223-b735-b5cdb4cfaed7 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: 197f2c5a-2a39-4bf1-af03-0d43f837482e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='133cfdea-39a1-4b23-8385-48901abd629e',volume_size=None,volume_type=None)] {{(pid=118352) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Aug 08 02:39:04.536308 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-2eac83c9-cb64-4223-b735-b5cdb4cfaed7 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.082391 Aug 08 02:39:04.540706 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 570/1131] 10.222.0.27 () {60 vars in 1147 bytes} [Mon Aug 8 02:39:03 2022] POST /compute/v2.1/servers => generated 406 bytes in 1084 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 08 02:39:04.547076 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_api:SELECT=36 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:39:04.548225 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell1:SELECT=64 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:39:04.552555 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-01b0718e-34c8-483f-86a4-2a9e72a825d3 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:39:04.553283 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-01b0718e-34c8-483f-86a4-2a9e72a825d3 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: 197f2c5a-2a39-4bf1-af03-0d43f837482e] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:39:04.609968 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-01b0718e-34c8-483f-86a4-2a9e72a825d3 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:39:04.616625 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-01b0718e-34c8-483f-86a4-2a9e72a825d3 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/197f2c5a-2a39-4bf1-af03-0d43f837482e" status: 200 len: 1095 microversion: 2.1 time: 0.072479 Aug 08 02:39:04.617217 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 562/1132] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:39:04 2022] GET /compute/v2.1/servers/197f2c5a-2a39-4bf1-af03-0d43f837482e => generated 1095 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:39:05.637473 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-dd365123-cd24-435d-8b2a-2b6cc75dc0ca tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:39:05.638236 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-dd365123-cd24-435d-8b2a-2b6cc75dc0ca tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: 197f2c5a-2a39-4bf1-af03-0d43f837482e] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:39:05.645015 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dd365123-cd24-435d-8b2a-2b6cc75dc0ca tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:39:05.645443 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dd365123-cd24-435d-8b2a-2b6cc75dc0ca tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:39:05.645874 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dd365123-cd24-435d-8b2a-2b6cc75dc0ca tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:39:05.646554 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dd365123-cd24-435d-8b2a-2b6cc75dc0ca tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:39:05.647003 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dd365123-cd24-435d-8b2a-2b6cc75dc0ca tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:39:05.647424 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-dd365123-cd24-435d-8b2a-2b6cc75dc0ca tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:39:05.711433 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-dd365123-cd24-435d-8b2a-2b6cc75dc0ca tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:39:05.719700 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-dd365123-cd24-435d-8b2a-2b6cc75dc0ca tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/197f2c5a-2a39-4bf1-af03-0d43f837482e" status: 200 len: 1145 microversion: 2.1 time: 0.086590 Aug 08 02:39:05.720256 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 571/1133] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:39:05 2022] GET /compute/v2.1/servers/197f2c5a-2a39-4bf1-af03-0d43f837482e => generated 1145 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:39:06.258500 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 08 02:39:06.259094 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Mon Aug 8 02:39:06 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 08 02:39:06.741725 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-cf7a1e5e-72ca-45db-a053-b83bcabe0b8a tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:39:06.742521 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-cf7a1e5e-72ca-45db-a053-b83bcabe0b8a tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: 197f2c5a-2a39-4bf1-af03-0d43f837482e] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:39:06.750782 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-cf7a1e5e-72ca-45db-a053-b83bcabe0b8a tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:39:06.751307 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-cf7a1e5e-72ca-45db-a053-b83bcabe0b8a tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:39:06.751739 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-cf7a1e5e-72ca-45db-a053-b83bcabe0b8a tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:39:06.752459 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-cf7a1e5e-72ca-45db-a053-b83bcabe0b8a tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:39:06.752934 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-cf7a1e5e-72ca-45db-a053-b83bcabe0b8a tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:39:06.753366 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-cf7a1e5e-72ca-45db-a053-b83bcabe0b8a tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:39:06.819005 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-cf7a1e5e-72ca-45db-a053-b83bcabe0b8a tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:39:06.827973 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-cf7a1e5e-72ca-45db-a053-b83bcabe0b8a tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/197f2c5a-2a39-4bf1-af03-0d43f837482e" status: 200 len: 1211 microversion: 2.1 time: 0.090315 Aug 08 02:39:06.828542 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 563/1134] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:39:06 2022] GET /compute/v2.1/servers/197f2c5a-2a39-4bf1-af03-0d43f837482e => generated 1211 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:39:07.855799 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-b97be314-e08b-4430-932d-f0030fb3ec6a tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:39:07.856418 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-b97be314-e08b-4430-932d-f0030fb3ec6a tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: 197f2c5a-2a39-4bf1-af03-0d43f837482e] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:39:07.863329 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b97be314-e08b-4430-932d-f0030fb3ec6a tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:39:07.863811 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b97be314-e08b-4430-932d-f0030fb3ec6a tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:39:07.864244 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b97be314-e08b-4430-932d-f0030fb3ec6a tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:39:07.864962 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b97be314-e08b-4430-932d-f0030fb3ec6a tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:39:07.865414 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b97be314-e08b-4430-932d-f0030fb3ec6a tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:39:07.865835 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-b97be314-e08b-4430-932d-f0030fb3ec6a tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:39:08.011584 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-b97be314-e08b-4430-932d-f0030fb3ec6a tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:39:08.020886 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-b97be314-e08b-4430-932d-f0030fb3ec6a tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/197f2c5a-2a39-4bf1-af03-0d43f837482e" status: 200 len: 1279 microversion: 2.1 time: 0.168935 Aug 08 02:39:08.026166 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 572/1135] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:39:07 2022] GET /compute/v2.1/servers/197f2c5a-2a39-4bf1-af03-0d43f837482e => generated 1279 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:39:08.232465 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [req-18700138-bd40-48ac-857a-d4dd8903f0c8 req-96d9d61a-1399-4c08-8548-f092ae374811 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "197f2c5a-2a39-4bf1-af03-0d43f837482e", "tag": "cb39a86c-07cf-4229-a5d2-7a8e2ccfa8af"}]} {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:39:08.242166 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-18700138-bd40-48ac-857a-d4dd8903f0c8 req-96d9d61a-1399-4c08-8548-f092ae374811 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:39:08.242646 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-18700138-bd40-48ac-857a-d4dd8903f0c8 req-96d9d61a-1399-4c08-8548-f092ae374811 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:39:08.243077 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [req-18700138-bd40-48ac-857a-d4dd8903f0c8 req-96d9d61a-1399-4c08-8548-f092ae374811 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:39:08.254520 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.compute.server_external_events [req-18700138-bd40-48ac-857a-d4dd8903f0c8 req-96d9d61a-1399-4c08-8548-f092ae374811 service nova] Creating event network-changed:cb39a86c-07cf-4229-a5d2-7a8e2ccfa8af for instance 197f2c5a-2a39-4bf1-af03-0d43f837482e on hv-cinder-74014 Aug 08 02:39:08.259294 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [req-18700138-bd40-48ac-857a-d4dd8903f0c8 req-96d9d61a-1399-4c08-8548-f092ae374811 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030729 Aug 08 02:39:08.259918 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 564/1136] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:39:08 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:39:09.041513 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-4f32a1f0-10f9-4c5a-81ad-8e25f5ec3fab tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:39:09.042129 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-4f32a1f0-10f9-4c5a-81ad-8e25f5ec3fab tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: 197f2c5a-2a39-4bf1-af03-0d43f837482e] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:39:09.049217 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4f32a1f0-10f9-4c5a-81ad-8e25f5ec3fab tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:39:09.050788 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4f32a1f0-10f9-4c5a-81ad-8e25f5ec3fab tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:39:09.050788 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4f32a1f0-10f9-4c5a-81ad-8e25f5ec3fab tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:39:09.050788 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4f32a1f0-10f9-4c5a-81ad-8e25f5ec3fab tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:39:09.051250 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4f32a1f0-10f9-4c5a-81ad-8e25f5ec3fab tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:39:09.051681 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4f32a1f0-10f9-4c5a-81ad-8e25f5ec3fab tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:39:09.181143 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-4f32a1f0-10f9-4c5a-81ad-8e25f5ec3fab tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:39:09.188590 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-4f32a1f0-10f9-4c5a-81ad-8e25f5ec3fab tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/197f2c5a-2a39-4bf1-af03-0d43f837482e" status: 200 len: 1267 microversion: 2.1 time: 0.150910 Aug 08 02:39:09.189183 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 573/1137] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:39:09 2022] GET /compute/v2.1/servers/197f2c5a-2a39-4bf1-af03-0d43f837482e => generated 1267 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:39:10.210078 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-97adb989-cb48-41e9-831e-e4a104ee6ac9 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:39:10.210771 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-97adb989-cb48-41e9-831e-e4a104ee6ac9 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: 197f2c5a-2a39-4bf1-af03-0d43f837482e] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:39:10.216929 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-97adb989-cb48-41e9-831e-e4a104ee6ac9 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:39:10.217406 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-97adb989-cb48-41e9-831e-e4a104ee6ac9 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:39:10.218809 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-97adb989-cb48-41e9-831e-e4a104ee6ac9 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:39:10.219493 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-97adb989-cb48-41e9-831e-e4a104ee6ac9 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:39:10.219949 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-97adb989-cb48-41e9-831e-e4a104ee6ac9 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:39:10.220372 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-97adb989-cb48-41e9-831e-e4a104ee6ac9 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:39:10.349163 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-97adb989-cb48-41e9-831e-e4a104ee6ac9 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:39:10.357103 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-97adb989-cb48-41e9-831e-e4a104ee6ac9 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/197f2c5a-2a39-4bf1-af03-0d43f837482e" status: 200 len: 1267 microversion: 2.1 time: 0.151593 Aug 08 02:39:10.357672 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 565/1138] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:39:10 2022] GET /compute/v2.1/servers/197f2c5a-2a39-4bf1-af03-0d43f837482e => generated 1267 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:39:11.378774 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-fff52ad5-8a40-43ee-95d5-8fe862f63a9d tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:39:11.379389 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-fff52ad5-8a40-43ee-95d5-8fe862f63a9d tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: 197f2c5a-2a39-4bf1-af03-0d43f837482e] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:39:11.385926 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-fff52ad5-8a40-43ee-95d5-8fe862f63a9d tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:39:11.386416 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-fff52ad5-8a40-43ee-95d5-8fe862f63a9d tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:39:11.386854 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-fff52ad5-8a40-43ee-95d5-8fe862f63a9d tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:39:11.387537 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-fff52ad5-8a40-43ee-95d5-8fe862f63a9d tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:39:11.387992 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-fff52ad5-8a40-43ee-95d5-8fe862f63a9d tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:39:11.388418 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-fff52ad5-8a40-43ee-95d5-8fe862f63a9d tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:39:11.518067 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-fff52ad5-8a40-43ee-95d5-8fe862f63a9d tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:39:11.525541 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-fff52ad5-8a40-43ee-95d5-8fe862f63a9d tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/197f2c5a-2a39-4bf1-af03-0d43f837482e" status: 200 len: 1267 microversion: 2.1 time: 0.150644 Aug 08 02:39:11.526132 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 574/1139] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:39:11 2022] GET /compute/v2.1/servers/197f2c5a-2a39-4bf1-af03-0d43f837482e => generated 1267 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:39:12.546908 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-6bbbebdf-a37e-412d-8f2f-11e7b8b874b5 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:39:12.547533 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-6bbbebdf-a37e-412d-8f2f-11e7b8b874b5 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: 197f2c5a-2a39-4bf1-af03-0d43f837482e] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:39:12.555784 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6bbbebdf-a37e-412d-8f2f-11e7b8b874b5 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:39:12.556691 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6bbbebdf-a37e-412d-8f2f-11e7b8b874b5 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:39:12.557151 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6bbbebdf-a37e-412d-8f2f-11e7b8b874b5 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:39:12.557878 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6bbbebdf-a37e-412d-8f2f-11e7b8b874b5 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:39:12.558338 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6bbbebdf-a37e-412d-8f2f-11e7b8b874b5 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:39:12.558771 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6bbbebdf-a37e-412d-8f2f-11e7b8b874b5 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:39:12.699518 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-6bbbebdf-a37e-412d-8f2f-11e7b8b874b5 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:39:12.707542 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-6bbbebdf-a37e-412d-8f2f-11e7b8b874b5 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/197f2c5a-2a39-4bf1-af03-0d43f837482e" status: 200 len: 1267 microversion: 2.1 time: 0.164691 Aug 08 02:39:12.708105 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 566/1140] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:39:12 2022] GET /compute/v2.1/servers/197f2c5a-2a39-4bf1-af03-0d43f837482e => generated 1267 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:39:13.729213 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-8cc51169-1b12-47be-85a1-4b0c7214a052 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:39:13.729756 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-8cc51169-1b12-47be-85a1-4b0c7214a052 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: 197f2c5a-2a39-4bf1-af03-0d43f837482e] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:39:13.736622 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8cc51169-1b12-47be-85a1-4b0c7214a052 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:39:13.737132 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8cc51169-1b12-47be-85a1-4b0c7214a052 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:39:13.737565 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8cc51169-1b12-47be-85a1-4b0c7214a052 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:39:13.738253 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8cc51169-1b12-47be-85a1-4b0c7214a052 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:39:13.738708 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8cc51169-1b12-47be-85a1-4b0c7214a052 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:39:13.739134 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8cc51169-1b12-47be-85a1-4b0c7214a052 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:39:13.871022 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-8cc51169-1b12-47be-85a1-4b0c7214a052 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:39:13.879317 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-8cc51169-1b12-47be-85a1-4b0c7214a052 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/197f2c5a-2a39-4bf1-af03-0d43f837482e" status: 200 len: 1267 microversion: 2.1 time: 0.154109 Aug 08 02:39:13.879884 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 575/1141] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:39:13 2022] GET /compute/v2.1/servers/197f2c5a-2a39-4bf1-af03-0d43f837482e => generated 1267 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:39:14.899735 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-6a09f7b9-84c2-44ef-9539-c26b938ed113 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:39:14.900398 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-6a09f7b9-84c2-44ef-9539-c26b938ed113 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: 197f2c5a-2a39-4bf1-af03-0d43f837482e] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:39:14.902875 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell0:SELECT=2 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:39:14.907835 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6a09f7b9-84c2-44ef-9539-c26b938ed113 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:39:14.908312 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6a09f7b9-84c2-44ef-9539-c26b938ed113 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:39:14.908769 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6a09f7b9-84c2-44ef-9539-c26b938ed113 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:39:14.909459 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6a09f7b9-84c2-44ef-9539-c26b938ed113 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:39:14.909909 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6a09f7b9-84c2-44ef-9539-c26b938ed113 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:39:14.910327 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6a09f7b9-84c2-44ef-9539-c26b938ed113 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:39:15.052407 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-6a09f7b9-84c2-44ef-9539-c26b938ed113 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:39:15.059906 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-6a09f7b9-84c2-44ef-9539-c26b938ed113 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/197f2c5a-2a39-4bf1-af03-0d43f837482e" status: 200 len: 1267 microversion: 2.1 time: 0.164549 Aug 08 02:39:15.060459 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 567/1142] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:39:14 2022] GET /compute/v2.1/servers/197f2c5a-2a39-4bf1-af03-0d43f837482e => generated 1267 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:39:16.082120 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-4f40a694-0fe1-4539-9e87-a53e6b3bf7f5 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:39:16.082731 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-4f40a694-0fe1-4539-9e87-a53e6b3bf7f5 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: 197f2c5a-2a39-4bf1-af03-0d43f837482e] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:39:16.084564 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell0:SELECT=2 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:39:16.085241 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell0:SELECT=6 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:39:16.090489 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4f40a694-0fe1-4539-9e87-a53e6b3bf7f5 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:39:16.096468 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4f40a694-0fe1-4539-9e87-a53e6b3bf7f5 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:39:16.096468 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4f40a694-0fe1-4539-9e87-a53e6b3bf7f5 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:39:16.096468 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4f40a694-0fe1-4539-9e87-a53e6b3bf7f5 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:39:16.096468 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4f40a694-0fe1-4539-9e87-a53e6b3bf7f5 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:39:16.096914 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-4f40a694-0fe1-4539-9e87-a53e6b3bf7f5 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:39:16.226935 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-4f40a694-0fe1-4539-9e87-a53e6b3bf7f5 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:39:16.234426 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-4f40a694-0fe1-4539-9e87-a53e6b3bf7f5 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/197f2c5a-2a39-4bf1-af03-0d43f837482e" status: 200 len: 1267 microversion: 2.1 time: 0.156473 Aug 08 02:39:16.234991 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 576/1143] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:39:16 2022] GET /compute/v2.1/servers/197f2c5a-2a39-4bf1-af03-0d43f837482e => generated 1267 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:39:17.306609 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-87341385-34b9-42f4-b161-24fd5baef66b tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:39:17.307225 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-87341385-34b9-42f4-b161-24fd5baef66b tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: 197f2c5a-2a39-4bf1-af03-0d43f837482e] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:39:17.313252 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-87341385-34b9-42f4-b161-24fd5baef66b tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:39:17.313732 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-87341385-34b9-42f4-b161-24fd5baef66b tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:39:17.314165 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-87341385-34b9-42f4-b161-24fd5baef66b tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:39:17.314843 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-87341385-34b9-42f4-b161-24fd5baef66b tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:39:17.315320 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-87341385-34b9-42f4-b161-24fd5baef66b tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:39:17.315738 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-87341385-34b9-42f4-b161-24fd5baef66b tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:39:17.446577 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-87341385-34b9-42f4-b161-24fd5baef66b tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:39:17.454634 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-87341385-34b9-42f4-b161-24fd5baef66b tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/197f2c5a-2a39-4bf1-af03-0d43f837482e" status: 200 len: 1267 microversion: 2.1 time: 0.203155 Aug 08 02:39:17.455201 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 568/1144] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:39:17 2022] GET /compute/v2.1/servers/197f2c5a-2a39-4bf1-af03-0d43f837482e => generated 1267 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:39:18.475409 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-d8e8b34b-5f95-4635-9886-8054103b8bf4 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:39:18.476695 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-d8e8b34b-5f95-4635-9886-8054103b8bf4 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: 197f2c5a-2a39-4bf1-af03-0d43f837482e] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:39:18.483151 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d8e8b34b-5f95-4635-9886-8054103b8bf4 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:39:18.483629 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d8e8b34b-5f95-4635-9886-8054103b8bf4 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:39:18.484062 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d8e8b34b-5f95-4635-9886-8054103b8bf4 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:39:18.485031 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d8e8b34b-5f95-4635-9886-8054103b8bf4 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:39:18.485334 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d8e8b34b-5f95-4635-9886-8054103b8bf4 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:39:18.485761 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-d8e8b34b-5f95-4635-9886-8054103b8bf4 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:39:18.616779 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-d8e8b34b-5f95-4635-9886-8054103b8bf4 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:39:18.624224 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-d8e8b34b-5f95-4635-9886-8054103b8bf4 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/197f2c5a-2a39-4bf1-af03-0d43f837482e" status: 200 len: 1267 microversion: 2.1 time: 0.152686 Aug 08 02:39:18.624816 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 577/1145] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:39:18 2022] GET /compute/v2.1/servers/197f2c5a-2a39-4bf1-af03-0d43f837482e => generated 1267 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:39:19.646490 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-7e9cd961-1175-44fe-9f77-f6f3eba24200 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:39:19.647092 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-7e9cd961-1175-44fe-9f77-f6f3eba24200 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: 197f2c5a-2a39-4bf1-af03-0d43f837482e] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:39:19.653417 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7e9cd961-1175-44fe-9f77-f6f3eba24200 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:39:19.653967 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7e9cd961-1175-44fe-9f77-f6f3eba24200 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:39:19.654438 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7e9cd961-1175-44fe-9f77-f6f3eba24200 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:39:19.655131 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7e9cd961-1175-44fe-9f77-f6f3eba24200 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:39:19.655585 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7e9cd961-1175-44fe-9f77-f6f3eba24200 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:39:19.656003 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-7e9cd961-1175-44fe-9f77-f6f3eba24200 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:39:19.794261 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-7e9cd961-1175-44fe-9f77-f6f3eba24200 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:39:19.802732 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-7e9cd961-1175-44fe-9f77-f6f3eba24200 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/197f2c5a-2a39-4bf1-af03-0d43f837482e" status: 200 len: 1267 microversion: 2.1 time: 0.160221 Aug 08 02:39:19.803294 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 569/1146] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:39:19 2022] GET /compute/v2.1/servers/197f2c5a-2a39-4bf1-af03-0d43f837482e => generated 1267 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:39:19.959679 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [req-49a201f5-84fc-4e7d-a581-3b1466fe4ead req-3c4df11e-30bb-4234-9df2-9359e01c99de service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "197f2c5a-2a39-4bf1-af03-0d43f837482e", "name": "network-vif-plugged", "status": "completed", "tag": "cb39a86c-07cf-4229-a5d2-7a8e2ccfa8af"}]} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:39:19.969861 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-49a201f5-84fc-4e7d-a581-3b1466fe4ead req-3c4df11e-30bb-4234-9df2-9359e01c99de service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:39:19.970335 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-49a201f5-84fc-4e7d-a581-3b1466fe4ead req-3c4df11e-30bb-4234-9df2-9359e01c99de service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:39:19.970758 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-49a201f5-84fc-4e7d-a581-3b1466fe4ead req-3c4df11e-30bb-4234-9df2-9359e01c99de service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:39:19.981906 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.compute.server_external_events [req-49a201f5-84fc-4e7d-a581-3b1466fe4ead req-3c4df11e-30bb-4234-9df2-9359e01c99de service nova] Creating event network-vif-plugged:cb39a86c-07cf-4229-a5d2-7a8e2ccfa8af for instance 197f2c5a-2a39-4bf1-af03-0d43f837482e on hv-cinder-74014 Aug 08 02:39:19.987448 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [req-49a201f5-84fc-4e7d-a581-3b1466fe4ead req-3c4df11e-30bb-4234-9df2-9359e01c99de service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031531 Aug 08 02:39:19.989077 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 578/1147] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:39:19 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:39:20.823225 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-033f1a87-50a0-402c-a824-32ac40c84b19 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:39:20.823848 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-033f1a87-50a0-402c-a824-32ac40c84b19 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: 197f2c5a-2a39-4bf1-af03-0d43f837482e] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:39:20.831037 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-033f1a87-50a0-402c-a824-32ac40c84b19 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:39:20.831521 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-033f1a87-50a0-402c-a824-32ac40c84b19 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:39:20.831956 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-033f1a87-50a0-402c-a824-32ac40c84b19 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:39:20.832644 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-033f1a87-50a0-402c-a824-32ac40c84b19 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:39:20.833125 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-033f1a87-50a0-402c-a824-32ac40c84b19 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:39:20.833552 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-033f1a87-50a0-402c-a824-32ac40c84b19 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:39:20.965254 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-033f1a87-50a0-402c-a824-32ac40c84b19 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:39:20.973193 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-033f1a87-50a0-402c-a824-32ac40c84b19 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/197f2c5a-2a39-4bf1-af03-0d43f837482e" status: 200 len: 1267 microversion: 2.1 time: 0.153805 Aug 08 02:39:20.973756 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 570/1148] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:39:20 2022] GET /compute/v2.1/servers/197f2c5a-2a39-4bf1-af03-0d43f837482e => generated 1267 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:39:21.995568 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-8516af14-e75a-4ebd-95d2-6e3684b9965b tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:39:21.996382 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-8516af14-e75a-4ebd-95d2-6e3684b9965b tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: 197f2c5a-2a39-4bf1-af03-0d43f837482e] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:39:22.004010 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8516af14-e75a-4ebd-95d2-6e3684b9965b tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:39:22.004491 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8516af14-e75a-4ebd-95d2-6e3684b9965b tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:39:22.004952 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8516af14-e75a-4ebd-95d2-6e3684b9965b tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:39:22.005707 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8516af14-e75a-4ebd-95d2-6e3684b9965b tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:39:22.006159 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8516af14-e75a-4ebd-95d2-6e3684b9965b tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:39:22.006581 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-8516af14-e75a-4ebd-95d2-6e3684b9965b tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:39:22.145172 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-8516af14-e75a-4ebd-95d2-6e3684b9965b tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:39:22.157296 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-8516af14-e75a-4ebd-95d2-6e3684b9965b tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/197f2c5a-2a39-4bf1-af03-0d43f837482e" status: 200 len: 1454 microversion: 2.1 time: 0.166001 Aug 08 02:39:22.158222 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 579/1149] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:39:21 2022] GET /compute/v2.1/servers/197f2c5a-2a39-4bf1-af03-0d43f837482e => generated 1454 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:39:22.183721 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-aeffc7cc-fb35-4739-8d91-d1bc58cb0e1c tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:39:22.184345 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-aeffc7cc-fb35-4739-8d91-d1bc58cb0e1c tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: 197f2c5a-2a39-4bf1-af03-0d43f837482e] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:39:22.191854 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-aeffc7cc-fb35-4739-8d91-d1bc58cb0e1c tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:39:22.192342 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-aeffc7cc-fb35-4739-8d91-d1bc58cb0e1c tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:39:22.192819 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-aeffc7cc-fb35-4739-8d91-d1bc58cb0e1c tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:39:22.193518 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-aeffc7cc-fb35-4739-8d91-d1bc58cb0e1c tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:39:22.193984 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-aeffc7cc-fb35-4739-8d91-d1bc58cb0e1c tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:39:22.194416 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-aeffc7cc-fb35-4739-8d91-d1bc58cb0e1c tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:39:22.548032 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-aeffc7cc-fb35-4739-8d91-d1bc58cb0e1c tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:39:22.555693 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-aeffc7cc-fb35-4739-8d91-d1bc58cb0e1c tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/197f2c5a-2a39-4bf1-af03-0d43f837482e" status: 200 len: 1454 microversion: 2.1 time: 0.378480 Aug 08 02:39:22.556254 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 571/1150] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:39:22 2022] GET /compute/v2.1/servers/197f2c5a-2a39-4bf1-af03-0d43f837482e => generated 1454 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:39:24.412187 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [req-b282c631-6f19-4e67-9b97-07e6740d9fab req-1d2811a7-79d0-4e1d-a4ea-fed4c9352e9d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "197f2c5a-2a39-4bf1-af03-0d43f837482e", "tag": "cb39a86c-07cf-4229-a5d2-7a8e2ccfa8af"}]} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:39:24.421842 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-b282c631-6f19-4e67-9b97-07e6740d9fab req-1d2811a7-79d0-4e1d-a4ea-fed4c9352e9d service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:39:24.422886 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-b282c631-6f19-4e67-9b97-07e6740d9fab req-1d2811a7-79d0-4e1d-a4ea-fed4c9352e9d service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:39:24.423317 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-b282c631-6f19-4e67-9b97-07e6740d9fab req-1d2811a7-79d0-4e1d-a4ea-fed4c9352e9d service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:39:24.434513 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.compute.server_external_events [req-b282c631-6f19-4e67-9b97-07e6740d9fab req-1d2811a7-79d0-4e1d-a4ea-fed4c9352e9d service nova] Creating event network-changed:cb39a86c-07cf-4229-a5d2-7a8e2ccfa8af for instance 197f2c5a-2a39-4bf1-af03-0d43f837482e on hv-cinder-74014 Aug 08 02:39:24.439404 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [req-b282c631-6f19-4e67-9b97-07e6740d9fab req-1d2811a7-79d0-4e1d-a4ea-fed4c9352e9d service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038292 Aug 08 02:39:24.449675 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 580/1151] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:39:24 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:40:25.810985 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 08 02:40:25.811682 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Mon Aug 8 02:40:25 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 08 02:40:32.770617 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 08 02:40:32.771224 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Mon Aug 8 02:40:32 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 08 02:40:53.351207 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_api:SELECT=35,nova_api:INSERT=3 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:40:53.352781 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-3b7110f7-29dc-4e42-bedc-30ab826920ed tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:40:53.353225 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG dbcounter [-] [118352] Writing DB stats nova_cell1:SELECT=54 {{(pid=118352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:40:53.353328 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-3b7110f7-29dc-4e42-bedc-30ab826920ed tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: 197f2c5a-2a39-4bf1-af03-0d43f837482e] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:40:53.356949 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_api:SELECT=22 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:40:53.359194 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG dbcounter [-] [118351] Writing DB stats nova_cell1:SELECT=50 {{(pid=118351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:40:53.359319 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [req-9eb570b2-e7b3-40bb-af92-1fdfc2061b19 req-dc4f4f76-4aba-487e-b59d-d466fdb889d8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "197f2c5a-2a39-4bf1-af03-0d43f837482e", "tag": "cb39a86c-07cf-4229-a5d2-7a8e2ccfa8af"}]} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:40:53.361250 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3b7110f7-29dc-4e42-bedc-30ab826920ed tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:40:53.361934 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3b7110f7-29dc-4e42-bedc-30ab826920ed tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:40:53.362649 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3b7110f7-29dc-4e42-bedc-30ab826920ed tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:40:53.363543 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3b7110f7-29dc-4e42-bedc-30ab826920ed tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:40:53.364171 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3b7110f7-29dc-4e42-bedc-30ab826920ed tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:40:53.364793 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-3b7110f7-29dc-4e42-bedc-30ab826920ed tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:40:53.368814 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-9eb570b2-e7b3-40bb-af92-1fdfc2061b19 req-dc4f4f76-4aba-487e-b59d-d466fdb889d8 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:40:53.369314 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-9eb570b2-e7b3-40bb-af92-1fdfc2061b19 req-dc4f4f76-4aba-487e-b59d-d466fdb889d8 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:40:53.369751 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-9eb570b2-e7b3-40bb-af92-1fdfc2061b19 req-dc4f4f76-4aba-487e-b59d-d466fdb889d8 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:40:53.381582 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.compute.server_external_events [req-9eb570b2-e7b3-40bb-af92-1fdfc2061b19 req-dc4f4f76-4aba-487e-b59d-d466fdb889d8 service nova] Creating event network-changed:cb39a86c-07cf-4229-a5d2-7a8e2ccfa8af for instance 197f2c5a-2a39-4bf1-af03-0d43f837482e on hv-cinder-74014 Aug 08 02:40:53.387270 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [req-9eb570b2-e7b3-40bb-af92-1fdfc2061b19 req-dc4f4f76-4aba-487e-b59d-d466fdb889d8 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037998 Aug 08 02:40:53.387922 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 581/1152] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:40:53 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 08 02:40:53.392819 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-3b7110f7-29dc-4e42-bedc-30ab826920ed tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: 197f2c5a-2a39-4bf1-af03-0d43f837482e] Going to try to terminate instance {{(pid=118351) delete /opt/stack/nova/nova/compute/api.py:2666}} Aug 08 02:40:53.733246 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-3b7110f7-29dc-4e42-bedc-30ab826920ed tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "DELETE /compute/v2.1/servers/197f2c5a-2a39-4bf1-af03-0d43f837482e" status: 204 len: 0 microversion: 2.1 time: 0.386374 Aug 08 02:40:53.734045 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 572/1153] 10.222.0.27 () {58 vars in 1239 bytes} [Mon Aug 8 02:40:53 2022] DELETE /compute/v2.1/servers/197f2c5a-2a39-4bf1-af03-0d43f837482e => generated 0 bytes in 388 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 08 02:40:53.750597 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-c9eb5ece-096a-4386-926d-14004fbc9067 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:40:53.752127 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-c9eb5ece-096a-4386-926d-14004fbc9067 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: 197f2c5a-2a39-4bf1-af03-0d43f837482e] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:40:53.760009 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c9eb5ece-096a-4386-926d-14004fbc9067 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:40:53.760816 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c9eb5ece-096a-4386-926d-14004fbc9067 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:40:53.761359 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c9eb5ece-096a-4386-926d-14004fbc9067 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:40:53.762267 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c9eb5ece-096a-4386-926d-14004fbc9067 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:40:53.762947 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c9eb5ece-096a-4386-926d-14004fbc9067 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:40:53.763439 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-c9eb5ece-096a-4386-926d-14004fbc9067 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:40:53.958940 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-c9eb5ece-096a-4386-926d-14004fbc9067 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:40:53.967365 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-c9eb5ece-096a-4386-926d-14004fbc9067 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/197f2c5a-2a39-4bf1-af03-0d43f837482e" status: 200 len: 1579 microversion: 2.1 time: 0.221662 Aug 08 02:40:53.967934 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 582/1154] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:40:53 2022] GET /compute/v2.1/servers/197f2c5a-2a39-4bf1-af03-0d43f837482e => generated 1579 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:40:54.989789 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-bdbbf926-a9aa-453f-9ae1-cb648e71a1aa tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:40:54.990484 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-bdbbf926-a9aa-453f-9ae1-cb648e71a1aa tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: 197f2c5a-2a39-4bf1-af03-0d43f837482e] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:40:54.997680 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-bdbbf926-a9aa-453f-9ae1-cb648e71a1aa tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:40:54.998157 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-bdbbf926-a9aa-453f-9ae1-cb648e71a1aa tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:40:54.998590 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-bdbbf926-a9aa-453f-9ae1-cb648e71a1aa tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:40:54.999269 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-bdbbf926-a9aa-453f-9ae1-cb648e71a1aa tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:40:54.999800 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-bdbbf926-a9aa-453f-9ae1-cb648e71a1aa tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:40:55.000237 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-bdbbf926-a9aa-453f-9ae1-cb648e71a1aa tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:40:55.395833 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-bdbbf926-a9aa-453f-9ae1-cb648e71a1aa tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:40:55.404432 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-bdbbf926-a9aa-453f-9ae1-cb648e71a1aa tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/197f2c5a-2a39-4bf1-af03-0d43f837482e" status: 200 len: 1460 microversion: 2.1 time: 0.418839 Aug 08 02:40:55.405037 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 573/1155] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:40:54 2022] GET /compute/v2.1/servers/197f2c5a-2a39-4bf1-af03-0d43f837482e => generated 1460 bytes in 420 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:40:55.741468 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [req-6ca8df5f-911b-4397-8615-302ad23a3530 req-884f59a6-5c49-4602-9613-0a42563b0e96 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "197f2c5a-2a39-4bf1-af03-0d43f837482e", "name": "network-vif-deleted", "tag": "cb39a86c-07cf-4229-a5d2-7a8e2ccfa8af"}]} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:40:55.751309 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-6ca8df5f-911b-4397-8615-302ad23a3530 req-884f59a6-5c49-4602-9613-0a42563b0e96 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:40:55.751791 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-6ca8df5f-911b-4397-8615-302ad23a3530 req-884f59a6-5c49-4602-9613-0a42563b0e96 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:40:55.752225 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-6ca8df5f-911b-4397-8615-302ad23a3530 req-884f59a6-5c49-4602-9613-0a42563b0e96 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:40:55.763521 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.compute.server_external_events [req-6ca8df5f-911b-4397-8615-302ad23a3530 req-884f59a6-5c49-4602-9613-0a42563b0e96 service nova] Creating event network-vif-deleted:cb39a86c-07cf-4229-a5d2-7a8e2ccfa8af for instance 197f2c5a-2a39-4bf1-af03-0d43f837482e on hv-cinder-74014 Aug 08 02:40:55.768261 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [req-6ca8df5f-911b-4397-8615-302ad23a3530 req-884f59a6-5c49-4602-9613-0a42563b0e96 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030416 Aug 08 02:40:55.768868 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 583/1156] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:40:55 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:40:56.425795 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-913a6935-7850-4e14-8670-2800da23303d tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:40:56.426340 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-913a6935-7850-4e14-8670-2800da23303d tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: 197f2c5a-2a39-4bf1-af03-0d43f837482e] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:40:56.433985 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-913a6935-7850-4e14-8670-2800da23303d tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:40:56.434464 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-913a6935-7850-4e14-8670-2800da23303d tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:40:56.435139 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-913a6935-7850-4e14-8670-2800da23303d tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:40:56.435830 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-913a6935-7850-4e14-8670-2800da23303d tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:40:56.436294 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-913a6935-7850-4e14-8670-2800da23303d tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:40:56.437187 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-913a6935-7850-4e14-8670-2800da23303d tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:40:56.467743 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-913a6935-7850-4e14-8670-2800da23303d tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:40:56.468148 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-913a6935-7850-4e14-8670-2800da23303d tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:40:56.468571 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-913a6935-7850-4e14-8670-2800da23303d tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:40:56.469031 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.objects.instance [None req-913a6935-7850-4e14-8670-2800da23303d tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lazy-loading 'fault' on Instance uuid 197f2c5a-2a39-4bf1-af03-0d43f837482e {{(pid=118351) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 08 02:40:56.509787 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-913a6935-7850-4e14-8670-2800da23303d tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:40:56.517463 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-913a6935-7850-4e14-8670-2800da23303d tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/197f2c5a-2a39-4bf1-af03-0d43f837482e" status: 200 len: 1231 microversion: 2.1 time: 0.095653 Aug 08 02:40:56.518021 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 574/1157] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:40:56 2022] GET /compute/v2.1/servers/197f2c5a-2a39-4bf1-af03-0d43f837482e => generated 1231 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:40:57.540064 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-3c1041c6-185e-4564-a1f8-a2df48a6815d tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:40:57.540848 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-3c1041c6-185e-4564-a1f8-a2df48a6815d tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: 197f2c5a-2a39-4bf1-af03-0d43f837482e] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:40:57.547066 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3c1041c6-185e-4564-a1f8-a2df48a6815d tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:40:57.547546 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3c1041c6-185e-4564-a1f8-a2df48a6815d tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:40:57.547982 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3c1041c6-185e-4564-a1f8-a2df48a6815d tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:40:57.548692 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3c1041c6-185e-4564-a1f8-a2df48a6815d tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:40:57.549145 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3c1041c6-185e-4564-a1f8-a2df48a6815d tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:40:57.549572 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3c1041c6-185e-4564-a1f8-a2df48a6815d tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:40:57.565398 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.wsgi [None req-3c1041c6-185e-4564-a1f8-a2df48a6815d tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] HTTP exception thrown: Instance 197f2c5a-2a39-4bf1-af03-0d43f837482e could not be found. Aug 08 02:40:57.566995 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-3c1041c6-185e-4564-a1f8-a2df48a6815d tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Request method failure captured: Aug 08 02:40:57.566995 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: request: GET /compute/v2.1/servers/197f2c5a-2a39-4bf1-af03-0d43f837482e HTTP/1.1 Aug 08 02:40:57.566995 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept: application/json Aug 08 02:40:57.566995 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept-Encoding: identity Aug 08 02:40:57.566995 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Connection: close Aug 08 02:40:57.566995 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Length: 0 Aug 08 02:40:57.566995 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Type: application/json Aug 08 02:40:57.566995 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Host: 10.222.0.27 Aug 08 02:40:57.566995 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Openstack-System-Scope: None Aug 08 02:40:57.566995 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: User-Agent: python-urllib3/1.26.11 Aug 08 02:40:57.566995 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Auth-Token: gAAAAABi8HYc3ITkcL195SgamaPJ0kidCrco7MQMBVI60W3dhhxQywziFpjko5BS7_5EtpN1OOpCyROL3y5wbY4pgEtWsgqrbP1z3EwAek9uoasyWggHXtdlr9JFso4IRqI1CEN56WsilnuCNo8We4lcFQ-SzMkX3bk_YaN8OPqytfLXsxqiG4E Aug 08 02:40:57.566995 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Id: None Aug 08 02:40:57.566995 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Name: None Aug 08 02:40:57.566995 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Identity-Status: Confirmed Aug 08 02:40:57.566995 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Is-Admin-Project: True Aug 08 02:40:57.566995 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Id: default Aug 08 02:40:57.566995 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Name: Default Aug 08 02:40:57.566995 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Id: 4e2c1907fad4499aae599d49e1d2bc80 Aug 08 02:40:57.566995 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Name: tempest-TestVolumeBootPattern-1114932292 Aug 08 02:40:57.566995 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Role: reader,member Aug 08 02:40:57.566995 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Roles: reader,member Aug 08 02:40:57.568758 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/4e2c1907fad4499aae599d49e1d2bc80"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/4e2c1907fad4499aae599d49e1d2bc80"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_4e2c1907fad4499aae599d49e1d2bc80", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/4e2c1907fad4499aae599d49e1d2bc80"}]}] Aug 08 02:40:57.568758 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant: tempest-TestVolumeBootPattern-1114932292 Aug 08 02:40:57.568758 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Id: 4e2c1907fad4499aae599d49e1d2bc80 Aug 08 02:40:57.568758 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Name: tempest-TestVolumeBootPattern-1114932292 Aug 08 02:40:57.568758 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User: tempest-TestVolumeBootPattern-1114932292-project Aug 08 02:40:57.568758 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Id: default Aug 08 02:40:57.568758 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Name: Default Aug 08 02:40:57.568758 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Id: b92e9b1d8c96434086055f58b119ed57 Aug 08 02:40:57.568758 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Name: tempest-TestVolumeBootPattern-1114932292-project Aug 08 02:40:57.568758 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: method: > Aug 08 02:40:57.568758 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: exception: Instance 197f2c5a-2a39-4bf1-af03-0d43f837482e could not be found. Aug 08 02:40:57.568758 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: action_args: {'id': '197f2c5a-2a39-4bf1-af03-0d43f837482e'} Aug 08 02:40:57.569847 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:40:57.569847 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:40:57.569847 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 08 02:40:57.569847 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 08 02:40:57.569847 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 08 02:40:57.569847 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 08 02:40:57.569847 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 08 02:40:57.569847 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 08 02:40:57.569847 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 08 02:40:57.569847 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 08 02:40:57.569847 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 197f2c5a-2a39-4bf1-af03-0d43f837482e could not be found. Aug 08 02:40:57.569847 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:40:57.569847 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:40:57.569847 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:40:57.569847 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:40:57.569847 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:40:57.569847 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:40:57.569847 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:40:57.569847 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:40:57.569847 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:40:57.571553 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:40:57.571553 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 08 02:40:57.571553 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 08 02:40:57.571553 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 08 02:40:57.571553 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 08 02:40:57.571553 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 08 02:40:57.571553 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:40:57.571553 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 197f2c5a-2a39-4bf1-af03-0d43f837482e could not be found. Aug 08 02:40:57.571553 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:40:57.571553 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:40:57.571553 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:40:57.571553 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:40:57.571553 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:40:57.571553 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:40:57.571553 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:40:57.571553 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:40:57.571553 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 197f2c5a-2a39-4bf1-af03-0d43f837482e could not be found. Aug 08 02:40:57.571553 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:40:57.573969 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-3c1041c6-185e-4564-a1f8-a2df48a6815d tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Returning 404 to user: Instance 197f2c5a-2a39-4bf1-af03-0d43f837482e could not be found. {{(pid=118352) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:40:57.573969 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-3c1041c6-185e-4564-a1f8-a2df48a6815d tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/197f2c5a-2a39-4bf1-af03-0d43f837482e" status: 404 len: 111 microversion: 2.1 time: 0.033554 Aug 08 02:40:57.573969 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 584/1158] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:40:57 2022] GET /compute/v2.1/servers/197f2c5a-2a39-4bf1-af03-0d43f837482e => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 08 02:41:00.889648 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-f486801d-e2ce-4fe2-aa0e-22e1359e78fa tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:41:00.890171 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-f486801d-e2ce-4fe2-aa0e-22e1359e78fa tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: fa7f9e9e-8095-49ae-aaab-363a38be7e81] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:41:00.891593 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [req-d8ea5e72-3e17-41fc-b76f-e15c553650c3 req-4fdadff5-6ea7-4c8f-80c2-18d3624d8c8a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fa7f9e9e-8095-49ae-aaab-363a38be7e81", "tag": "4d9e69b4-4e91-42c2-8eab-03606390a0b6"}]} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:41:00.897047 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f486801d-e2ce-4fe2-aa0e-22e1359e78fa tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:41:00.897532 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f486801d-e2ce-4fe2-aa0e-22e1359e78fa tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:41:00.898021 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f486801d-e2ce-4fe2-aa0e-22e1359e78fa tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:41:00.898718 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f486801d-e2ce-4fe2-aa0e-22e1359e78fa tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:41:00.899233 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f486801d-e2ce-4fe2-aa0e-22e1359e78fa tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:41:00.899662 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f486801d-e2ce-4fe2-aa0e-22e1359e78fa tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:41:00.903234 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-d8ea5e72-3e17-41fc-b76f-e15c553650c3 req-4fdadff5-6ea7-4c8f-80c2-18d3624d8c8a service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:41:00.903967 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-d8ea5e72-3e17-41fc-b76f-e15c553650c3 req-4fdadff5-6ea7-4c8f-80c2-18d3624d8c8a service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:41:00.904625 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-d8ea5e72-3e17-41fc-b76f-e15c553650c3 req-4fdadff5-6ea7-4c8f-80c2-18d3624d8c8a service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:41:00.915983 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.compute.server_external_events [req-d8ea5e72-3e17-41fc-b76f-e15c553650c3 req-4fdadff5-6ea7-4c8f-80c2-18d3624d8c8a service nova] Creating event network-changed:4d9e69b4-4e91-42c2-8eab-03606390a0b6 for instance fa7f9e9e-8095-49ae-aaab-363a38be7e81 on hv-cinder-74014 Aug 08 02:41:00.926343 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [req-d8ea5e72-3e17-41fc-b76f-e15c553650c3 req-4fdadff5-6ea7-4c8f-80c2-18d3624d8c8a service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038397 Aug 08 02:41:00.926970 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-f486801d-e2ce-4fe2-aa0e-22e1359e78fa tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: fa7f9e9e-8095-49ae-aaab-363a38be7e81] Going to try to terminate instance {{(pid=118351) delete /opt/stack/nova/nova/compute/api.py:2666}} Aug 08 02:41:00.931386 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 585/1159] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:41:00 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:41:01.238653 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-f486801d-e2ce-4fe2-aa0e-22e1359e78fa tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "DELETE /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81" status: 204 len: 0 microversion: 2.1 time: 0.353496 Aug 08 02:41:01.239374 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 575/1160] 10.222.0.27 () {58 vars in 1239 bytes} [Mon Aug 8 02:41:00 2022] DELETE /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 08 02:41:01.253339 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-32d0794f-a42e-4e8b-a065-7a18bc45b6a6 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:41:01.253339 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-32d0794f-a42e-4e8b-a065-7a18bc45b6a6 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: fa7f9e9e-8095-49ae-aaab-363a38be7e81] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:41:01.256774 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-32d0794f-a42e-4e8b-a065-7a18bc45b6a6 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:41:01.257258 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-32d0794f-a42e-4e8b-a065-7a18bc45b6a6 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:41:01.257931 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-32d0794f-a42e-4e8b-a065-7a18bc45b6a6 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:41:01.258623 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-32d0794f-a42e-4e8b-a065-7a18bc45b6a6 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:41:01.259180 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-32d0794f-a42e-4e8b-a065-7a18bc45b6a6 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:41:01.259603 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-32d0794f-a42e-4e8b-a065-7a18bc45b6a6 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:41:01.411410 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.policy [None req-32d0794f-a42e-4e8b-a065-7a18bc45b6a6 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118352) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:41:01.419251 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-32d0794f-a42e-4e8b-a065-7a18bc45b6a6 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81" status: 200 len: 1577 microversion: 2.1 time: 0.175268 Aug 08 02:41:01.422614 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 586/1161] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:41:01 2022] GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81 => generated 1577 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:41:02.442203 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-f384b189-7275-4c71-aa65-be6328dcb5f8 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:41:02.442945 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-f384b189-7275-4c71-aa65-be6328dcb5f8 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: fa7f9e9e-8095-49ae-aaab-363a38be7e81] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:41:02.450737 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f384b189-7275-4c71-aa65-be6328dcb5f8 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:41:02.451358 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f384b189-7275-4c71-aa65-be6328dcb5f8 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:41:02.455114 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f384b189-7275-4c71-aa65-be6328dcb5f8 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:41:02.455114 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f384b189-7275-4c71-aa65-be6328dcb5f8 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:41:02.455114 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f384b189-7275-4c71-aa65-be6328dcb5f8 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:41:02.455114 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-f384b189-7275-4c71-aa65-be6328dcb5f8 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:41:02.597179 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-f384b189-7275-4c71-aa65-be6328dcb5f8 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:41:02.605184 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-f384b189-7275-4c71-aa65-be6328dcb5f8 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81" status: 200 len: 1577 microversion: 2.1 time: 0.167288 Aug 08 02:41:02.605737 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 576/1162] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:41:02 2022] GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81 => generated 1577 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:41:03.200187 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [req-8ee7cd69-9117-4432-b0f3-acc235a8d20c req-6bcb1c06-7f00-4b12-a9f2-aceb48fc1e38 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fa7f9e9e-8095-49ae-aaab-363a38be7e81", "name": "network-vif-deleted", "tag": "4d9e69b4-4e91-42c2-8eab-03606390a0b6"}]} {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 08 02:41:03.210436 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-8ee7cd69-9117-4432-b0f3-acc235a8d20c req-6bcb1c06-7f00-4b12-a9f2-aceb48fc1e38 service nova] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:41:03.217528 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-8ee7cd69-9117-4432-b0f3-acc235a8d20c req-6bcb1c06-7f00-4b12-a9f2-aceb48fc1e38 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:41:03.217528 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [req-8ee7cd69-9117-4432-b0f3-acc235a8d20c req-6bcb1c06-7f00-4b12-a9f2-aceb48fc1e38 service nova] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:41:03.222250 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.compute.server_external_events [req-8ee7cd69-9117-4432-b0f3-acc235a8d20c req-6bcb1c06-7f00-4b12-a9f2-aceb48fc1e38 service nova] Creating event network-vif-deleted:4d9e69b4-4e91-42c2-8eab-03606390a0b6 for instance fa7f9e9e-8095-49ae-aaab-363a38be7e81 on hv-cinder-74014 Aug 08 02:41:03.227075 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [req-8ee7cd69-9117-4432-b0f3-acc235a8d20c req-6bcb1c06-7f00-4b12-a9f2-aceb48fc1e38 service nova] 10.222.0.27 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030640 Aug 08 02:41:03.227651 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 587/1163] 10.222.0.27 () {64 vars in 1317 bytes} [Mon Aug 8 02:41:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 08 02:41:03.627628 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-d6b10cb6-0282-468f-87be-d078dc383fbb tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:41:03.628315 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-d6b10cb6-0282-468f-87be-d078dc383fbb tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: fa7f9e9e-8095-49ae-aaab-363a38be7e81] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:41:03.635086 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d6b10cb6-0282-468f-87be-d078dc383fbb tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:41:03.638403 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d6b10cb6-0282-468f-87be-d078dc383fbb tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:41:03.638403 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d6b10cb6-0282-468f-87be-d078dc383fbb tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:41:03.638403 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d6b10cb6-0282-468f-87be-d078dc383fbb tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:41:03.638403 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d6b10cb6-0282-468f-87be-d078dc383fbb tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:41:03.639071 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-d6b10cb6-0282-468f-87be-d078dc383fbb tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:41:03.701755 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.policy [None req-d6b10cb6-0282-468f-87be-d078dc383fbb tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b92e9b1d8c96434086055f58b119ed57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4e2c1907fad4499aae599d49e1d2bc80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118351) authorize /opt/stack/nova/nova/policy.py:199}} Aug 08 02:41:03.709795 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-d6b10cb6-0282-468f-87be-d078dc383fbb tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81" status: 200 len: 1225 microversion: 2.1 time: 0.086265 Aug 08 02:41:03.710363 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 577/1164] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:41:03 2022] GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81 => generated 1225 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 08 02:41:04.733180 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-3efcf413-f929-4aad-bebf-b22341a723f0 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:41:04.733880 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-3efcf413-f929-4aad-bebf-b22341a723f0 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: fa7f9e9e-8095-49ae-aaab-363a38be7e81] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:41:04.739999 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3efcf413-f929-4aad-bebf-b22341a723f0 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:41:04.740475 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3efcf413-f929-4aad-bebf-b22341a723f0 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:41:04.740937 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3efcf413-f929-4aad-bebf-b22341a723f0 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:41:04.741626 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3efcf413-f929-4aad-bebf-b22341a723f0 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:41:04.742086 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3efcf413-f929-4aad-bebf-b22341a723f0 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:41:04.742541 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-3efcf413-f929-4aad-bebf-b22341a723f0 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:41:04.755640 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.wsgi [None req-3efcf413-f929-4aad-bebf-b22341a723f0 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] HTTP exception thrown: Instance fa7f9e9e-8095-49ae-aaab-363a38be7e81 could not be found. Aug 08 02:41:04.757257 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-3efcf413-f929-4aad-bebf-b22341a723f0 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Request method failure captured: Aug 08 02:41:04.757257 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: request: GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81 HTTP/1.1 Aug 08 02:41:04.757257 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept: application/json Aug 08 02:41:04.757257 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept-Encoding: identity Aug 08 02:41:04.757257 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Connection: close Aug 08 02:41:04.757257 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Length: 0 Aug 08 02:41:04.757257 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Type: application/json Aug 08 02:41:04.757257 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Host: 10.222.0.27 Aug 08 02:41:04.757257 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Openstack-System-Scope: None Aug 08 02:41:04.757257 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: User-Agent: python-urllib3/1.26.11 Aug 08 02:41:04.757257 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Auth-Token: gAAAAABi8HYc3ITkcL195SgamaPJ0kidCrco7MQMBVI60W3dhhxQywziFpjko5BS7_5EtpN1OOpCyROL3y5wbY4pgEtWsgqrbP1z3EwAek9uoasyWggHXtdlr9JFso4IRqI1CEN56WsilnuCNo8We4lcFQ-SzMkX3bk_YaN8OPqytfLXsxqiG4E Aug 08 02:41:04.757257 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Id: None Aug 08 02:41:04.757257 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Name: None Aug 08 02:41:04.757257 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Identity-Status: Confirmed Aug 08 02:41:04.757257 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Is-Admin-Project: True Aug 08 02:41:04.757257 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Id: default Aug 08 02:41:04.757257 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Name: Default Aug 08 02:41:04.757257 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Id: 4e2c1907fad4499aae599d49e1d2bc80 Aug 08 02:41:04.757257 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Name: tempest-TestVolumeBootPattern-1114932292 Aug 08 02:41:04.757257 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Role: reader,member Aug 08 02:41:04.757257 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Roles: reader,member Aug 08 02:41:04.759062 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/4e2c1907fad4499aae599d49e1d2bc80"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/4e2c1907fad4499aae599d49e1d2bc80"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_4e2c1907fad4499aae599d49e1d2bc80", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/4e2c1907fad4499aae599d49e1d2bc80"}]}] Aug 08 02:41:04.759062 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant: tempest-TestVolumeBootPattern-1114932292 Aug 08 02:41:04.759062 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Id: 4e2c1907fad4499aae599d49e1d2bc80 Aug 08 02:41:04.759062 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Name: tempest-TestVolumeBootPattern-1114932292 Aug 08 02:41:04.759062 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User: tempest-TestVolumeBootPattern-1114932292-project Aug 08 02:41:04.759062 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Id: default Aug 08 02:41:04.759062 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Name: Default Aug 08 02:41:04.759062 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Id: b92e9b1d8c96434086055f58b119ed57 Aug 08 02:41:04.759062 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Name: tempest-TestVolumeBootPattern-1114932292-project Aug 08 02:41:04.759062 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: method: > Aug 08 02:41:04.759062 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: exception: Instance fa7f9e9e-8095-49ae-aaab-363a38be7e81 could not be found. Aug 08 02:41:04.759062 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: action_args: {'id': 'fa7f9e9e-8095-49ae-aaab-363a38be7e81'} Aug 08 02:41:04.760797 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:41:04.760797 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:41:04.760797 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 08 02:41:04.760797 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 08 02:41:04.760797 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 08 02:41:04.760797 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 08 02:41:04.760797 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 08 02:41:04.760797 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 08 02:41:04.760797 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 08 02:41:04.760797 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 08 02:41:04.760797 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fa7f9e9e-8095-49ae-aaab-363a38be7e81 could not be found. Aug 08 02:41:04.760797 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:41:04.760797 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:41:04.760797 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:41:04.760797 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:41:04.760797 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:41:04.760797 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:41:04.760797 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:41:04.760797 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:41:04.760797 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:41:04.763120 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:41:04.763120 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 08 02:41:04.763120 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 08 02:41:04.763120 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 08 02:41:04.763120 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 08 02:41:04.763120 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 08 02:41:04.763120 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:41:04.763120 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fa7f9e9e-8095-49ae-aaab-363a38be7e81 could not be found. Aug 08 02:41:04.763120 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:41:04.763120 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:41:04.763120 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:41:04.763120 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:41:04.763120 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:41:04.763120 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:41:04.763120 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:41:04.763120 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:41:04.763120 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fa7f9e9e-8095-49ae-aaab-363a38be7e81 could not be found. Aug 08 02:41:04.763120 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:41:04.765286 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-3efcf413-f929-4aad-bebf-b22341a723f0 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Returning 404 to user: Instance fa7f9e9e-8095-49ae-aaab-363a38be7e81 could not be found. {{(pid=118352) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:41:04.765286 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-3efcf413-f929-4aad-bebf-b22341a723f0 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81" status: 404 len: 111 microversion: 2.1 time: 0.030544 Aug 08 02:41:04.765286 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 588/1165] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:41:04 2022] GET /compute/v2.1/servers/fa7f9e9e-8095-49ae-aaab-363a38be7e81 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 08 02:41:05.261773 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-6feefec0-aa8a-4b69-b6ea-6e675ddacf63 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:41:05.262895 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.compute.api [None req-6feefec0-aa8a-4b69-b6ea-6e675ddacf63 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: f0d7098c-da43-4381-99e9-b4756dde20c8] Fetching instance by UUID {{(pid=118351) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:41:05.269380 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6feefec0-aa8a-4b69-b6ea-6e675ddacf63 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:41:05.270118 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6feefec0-aa8a-4b69-b6ea-6e675ddacf63 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:41:05.270341 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6feefec0-aa8a-4b69-b6ea-6e675ddacf63 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:41:05.271036 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6feefec0-aa8a-4b69-b6ea-6e675ddacf63 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:41:05.271488 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6feefec0-aa8a-4b69-b6ea-6e675ddacf63 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:41:05.271913 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG oslo_concurrency.lockutils [None req-6feefec0-aa8a-4b69-b6ea-6e675ddacf63 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118351) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:41:05.285265 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.wsgi [None req-6feefec0-aa8a-4b69-b6ea-6e675ddacf63 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] HTTP exception thrown: Instance f0d7098c-da43-4381-99e9-b4756dde20c8 could not be found. Aug 08 02:41:05.286951 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-6feefec0-aa8a-4b69-b6ea-6e675ddacf63 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Request method failure captured: Aug 08 02:41:05.286951 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: request: DELETE /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8 HTTP/1.1 Aug 08 02:41:05.286951 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Accept: application/json Aug 08 02:41:05.286951 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Accept-Encoding: identity Aug 08 02:41:05.286951 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Connection: close Aug 08 02:41:05.286951 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Content-Length: 0 Aug 08 02:41:05.286951 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Content-Type: application/json Aug 08 02:41:05.286951 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Host: 10.222.0.27 Aug 08 02:41:05.286951 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Openstack-System-Scope: None Aug 08 02:41:05.286951 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: User-Agent: python-urllib3/1.26.11 Aug 08 02:41:05.286951 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Auth-Token: gAAAAABi8HYc3ITkcL195SgamaPJ0kidCrco7MQMBVI60W3dhhxQywziFpjko5BS7_5EtpN1OOpCyROL3y5wbY4pgEtWsgqrbP1z3EwAek9uoasyWggHXtdlr9JFso4IRqI1CEN56WsilnuCNo8We4lcFQ-SzMkX3bk_YaN8OPqytfLXsxqiG4E Aug 08 02:41:05.286951 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Domain-Id: None Aug 08 02:41:05.286951 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Domain-Name: None Aug 08 02:41:05.286951 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Identity-Status: Confirmed Aug 08 02:41:05.286951 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Is-Admin-Project: True Aug 08 02:41:05.286951 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Domain-Id: default Aug 08 02:41:05.286951 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Domain-Name: Default Aug 08 02:41:05.286951 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Id: 4e2c1907fad4499aae599d49e1d2bc80 Aug 08 02:41:05.286951 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Project-Name: tempest-TestVolumeBootPattern-1114932292 Aug 08 02:41:05.286951 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Role: reader,member Aug 08 02:41:05.286951 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Roles: reader,member Aug 08 02:41:05.288771 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/4e2c1907fad4499aae599d49e1d2bc80"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/4e2c1907fad4499aae599d49e1d2bc80"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_4e2c1907fad4499aae599d49e1d2bc80", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/4e2c1907fad4499aae599d49e1d2bc80"}]}] Aug 08 02:41:05.288771 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant: tempest-TestVolumeBootPattern-1114932292 Aug 08 02:41:05.288771 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant-Id: 4e2c1907fad4499aae599d49e1d2bc80 Aug 08 02:41:05.288771 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-Tenant-Name: tempest-TestVolumeBootPattern-1114932292 Aug 08 02:41:05.288771 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User: tempest-TestVolumeBootPattern-1114932292-project Aug 08 02:41:05.288771 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Domain-Id: default Aug 08 02:41:05.288771 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Domain-Name: Default Aug 08 02:41:05.288771 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Id: b92e9b1d8c96434086055f58b119ed57 Aug 08 02:41:05.288771 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: X-User-Name: tempest-TestVolumeBootPattern-1114932292-project Aug 08 02:41:05.288771 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: method: > Aug 08 02:41:05.288771 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: exception: Instance f0d7098c-da43-4381-99e9-b4756dde20c8 could not be found. Aug 08 02:41:05.288771 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: action_args: {'id': 'f0d7098c-da43-4381-99e9-b4756dde20c8'} Aug 08 02:41:05.289751 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:41:05.289751 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:41:05.289751 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 08 02:41:05.289751 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 08 02:41:05.289751 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 08 02:41:05.289751 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 08 02:41:05.289751 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 08 02:41:05.289751 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 08 02:41:05.289751 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 08 02:41:05.289751 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 08 02:41:05.289751 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f0d7098c-da43-4381-99e9-b4756dde20c8 could not be found. Aug 08 02:41:05.289751 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:41:05.289751 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:41:05.289751 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:41:05.289751 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:41:05.289751 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:41:05.289751 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:41:05.289751 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:41:05.289751 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:41:05.289751 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:41:05.291492 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:41:05.291492 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 08 02:41:05.291492 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 08 02:41:05.291492 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 08 02:41:05.291492 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 08 02:41:05.291492 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 08 02:41:05.291492 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 08 02:41:05.291492 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 08 02:41:05.291492 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:41:05.291492 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f0d7098c-da43-4381-99e9-b4756dde20c8 could not be found. Aug 08 02:41:05.291492 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:41:05.291492 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:41:05.291492 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:41:05.291492 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:41:05.291492 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:41:05.291492 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:41:05.291492 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:41:05.291492 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:41:05.291492 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f0d7098c-da43-4381-99e9-b4756dde20c8 could not be found. Aug 08 02:41:05.291492 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: ERROR nova.api.openstack.wsgi Aug 08 02:41:05.293186 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-6feefec0-aa8a-4b69-b6ea-6e675ddacf63 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Returning 404 to user: Instance f0d7098c-da43-4381-99e9-b4756dde20c8 could not be found. {{(pid=118351) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:41:05.293186 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-6feefec0-aa8a-4b69-b6ea-6e675ddacf63 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "DELETE /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8" status: 404 len: 111 microversion: 2.1 time: 0.030485 Aug 08 02:41:05.293186 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 578/1166] 10.222.0.27 () {58 vars in 1239 bytes} [Mon Aug 8 02:41:05 2022] DELETE /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 08 02:41:05.297504 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-35338787-abc9-41e7-aa14-20ae558900a7 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '>' {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:41:05.298141 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.compute.api [None req-35338787-abc9-41e7-aa14-20ae558900a7 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] [instance: f0d7098c-da43-4381-99e9-b4756dde20c8] Fetching instance by UUID {{(pid=118352) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 08 02:41:05.304042 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-35338787-abc9-41e7-aa14-20ae558900a7 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:41:05.304516 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-35338787-abc9-41e7-aa14-20ae558900a7 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:41:05.304994 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-35338787-abc9-41e7-aa14-20ae558900a7 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:41:05.305673 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-35338787-abc9-41e7-aa14-20ae558900a7 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Acquiring lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 08 02:41:05.306122 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-35338787-abc9-41e7-aa14-20ae558900a7 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 08 02:41:05.306539 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG oslo_concurrency.lockutils [None req-35338787-abc9-41e7-aa14-20ae558900a7 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Lock "3f4309de-b8ab-46d7-9a8f-20127e13d27a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118352) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 08 02:41:05.320101 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.wsgi [None req-35338787-abc9-41e7-aa14-20ae558900a7 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] HTTP exception thrown: Instance f0d7098c-da43-4381-99e9-b4756dde20c8 could not be found. Aug 08 02:41:05.321609 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-35338787-abc9-41e7-aa14-20ae558900a7 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Request method failure captured: Aug 08 02:41:05.321609 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: request: GET /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8 HTTP/1.1 Aug 08 02:41:05.321609 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept: application/json Aug 08 02:41:05.321609 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Accept-Encoding: identity Aug 08 02:41:05.321609 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Connection: close Aug 08 02:41:05.321609 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Length: 0 Aug 08 02:41:05.321609 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Content-Type: application/json Aug 08 02:41:05.321609 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Host: 10.222.0.27 Aug 08 02:41:05.321609 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Openstack-System-Scope: None Aug 08 02:41:05.321609 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: User-Agent: python-urllib3/1.26.11 Aug 08 02:41:05.321609 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Auth-Token: gAAAAABi8HYc3ITkcL195SgamaPJ0kidCrco7MQMBVI60W3dhhxQywziFpjko5BS7_5EtpN1OOpCyROL3y5wbY4pgEtWsgqrbP1z3EwAek9uoasyWggHXtdlr9JFso4IRqI1CEN56WsilnuCNo8We4lcFQ-SzMkX3bk_YaN8OPqytfLXsxqiG4E Aug 08 02:41:05.321609 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Id: None Aug 08 02:41:05.321609 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Domain-Name: None Aug 08 02:41:05.321609 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Identity-Status: Confirmed Aug 08 02:41:05.321609 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Is-Admin-Project: True Aug 08 02:41:05.321609 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Id: default Aug 08 02:41:05.321609 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Domain-Name: Default Aug 08 02:41:05.321609 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Id: 4e2c1907fad4499aae599d49e1d2bc80 Aug 08 02:41:05.321609 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Project-Name: tempest-TestVolumeBootPattern-1114932292 Aug 08 02:41:05.321609 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Role: reader,member Aug 08 02:41:05.321609 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Roles: reader,member Aug 08 02:41:05.323489 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2/4e2c1907fad4499aae599d49e1d2bc80"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/4e2c1907fad4499aae599d49e1d2bc80"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:8080/v1/AUTH_4e2c1907fad4499aae599d49e1d2bc80", "adminURL": "http://10.222.0.27:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.27/volume/v3/4e2c1907fad4499aae599d49e1d2bc80"}]}] Aug 08 02:41:05.323489 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant: tempest-TestVolumeBootPattern-1114932292 Aug 08 02:41:05.323489 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Id: 4e2c1907fad4499aae599d49e1d2bc80 Aug 08 02:41:05.323489 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-Tenant-Name: tempest-TestVolumeBootPattern-1114932292 Aug 08 02:41:05.323489 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User: tempest-TestVolumeBootPattern-1114932292-project Aug 08 02:41:05.323489 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Id: default Aug 08 02:41:05.323489 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Domain-Name: Default Aug 08 02:41:05.323489 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Id: b92e9b1d8c96434086055f58b119ed57 Aug 08 02:41:05.323489 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: X-User-Name: tempest-TestVolumeBootPattern-1114932292-project Aug 08 02:41:05.323489 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: method: > Aug 08 02:41:05.323489 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: exception: Instance f0d7098c-da43-4381-99e9-b4756dde20c8 could not be found. Aug 08 02:41:05.323489 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: action_args: {'id': 'f0d7098c-da43-4381-99e9-b4756dde20c8'} Aug 08 02:41:05.324469 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: {{(pid=118352) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 08 02:41:05.324469 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:41:05.324469 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 08 02:41:05.324469 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 08 02:41:05.324469 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 08 02:41:05.324469 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 08 02:41:05.324469 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 08 02:41:05.324469 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 08 02:41:05.324469 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 08 02:41:05.324469 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 08 02:41:05.324469 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f0d7098c-da43-4381-99e9-b4756dde20c8 could not be found. Aug 08 02:41:05.324469 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:41:05.324469 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:41:05.324469 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:41:05.324469 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:41:05.324469 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:41:05.324469 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:41:05.324469 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 08 02:41:05.324469 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 08 02:41:05.324469 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 08 02:41:05.326222 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 08 02:41:05.326222 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 08 02:41:05.326222 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 08 02:41:05.326222 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 08 02:41:05.326222 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 08 02:41:05.326222 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 08 02:41:05.326222 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 08 02:41:05.326222 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f0d7098c-da43-4381-99e9-b4756dde20c8 could not be found. Aug 08 02:41:05.326222 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:41:05.326222 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 08 02:41:05.326222 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:41:05.326222 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 08 02:41:05.326222 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 08 02:41:05.326222 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 08 02:41:05.326222 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 08 02:41:05.326222 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 08 02:41:05.326222 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f0d7098c-da43-4381-99e9-b4756dde20c8 could not be found. Aug 08 02:41:05.326222 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: ERROR nova.api.openstack.wsgi Aug 08 02:41:05.327722 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: DEBUG nova.api.openstack.wsgi [None req-35338787-abc9-41e7-aa14-20ae558900a7 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Returning 404 to user: Instance f0d7098c-da43-4381-99e9-b4756dde20c8 could not be found. {{(pid=118352) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 08 02:41:05.327722 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO nova.api.openstack.requestlog [None req-35338787-abc9-41e7-aa14-20ae558900a7 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "GET /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8" status: 404 len: 111 microversion: 2.1 time: 0.029905 Aug 08 02:41:05.327722 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: [pid: 118352|app: 0|req: 589/1167] 10.222.0.27 () {58 vars in 1236 bytes} [Mon Aug 8 02:41:05 2022] GET /compute/v2.1/servers/f0d7098c-da43-4381-99e9-b4756dde20c8 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 08 02:41:06.800748 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: DEBUG nova.api.openstack.wsgi [None req-1452dc83-f81e-44f3-a59c-8fd0c92a80b9 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] Calling method '.version_select at 0x7efc74185280>' {{(pid=118351) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 08 02:41:06.896878 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO nova.api.openstack.requestlog [None req-1452dc83-f81e-44f3-a59c-8fd0c92a80b9 tempest-TestVolumeBootPattern-1114932292 tempest-TestVolumeBootPattern-1114932292-project] 10.222.0.27 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1683228822" status: 202 len: 0 microversion: 2.1 time: 0.100090 Aug 08 02:41:06.897594 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: [pid: 118351|app: 0|req: 579/1168] 10.222.0.27 () {58 vars in 1263 bytes} [Mon Aug 8 02:41:06 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1683228822 => generated 0 bytes in 101 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 08 02:42:18.348873 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 08 02:42:18.349408 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Mon Aug 8 02:42:18 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 08 02:43:45.819503 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 08 02:43:45.820195 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Mon Aug 8 02:43:45 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 08 02:43:52.779288 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 08 02:43:52.779767 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Mon Aug 8 02:43:52 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 08 02:44:06.268718 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 08 02:44:06.269522 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Mon Aug 8 02:44:06 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 08 02:44:08.372262 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 08 02:44:08.372958 dv-cinder-740146-18-iscsi devstack@n-api.service[118351]: Mon Aug 8 02:44:08 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 08 02:44:16.836473 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 08 02:44:16.837677 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Mon Aug 8 02:44:16 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 08 02:45:38.357055 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 08 02:45:38.357759 dv-cinder-740146-18-iscsi devstack@n-api.service[118352]: Mon Aug 8 02:45:38 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!!